JEP 197: Segmented Code CacheJEP 197: Segmented Code CacheOwnerTobias HartmannTypeFeatureScopeImplementationStatusClosed / DeliveredRelease9Componenthotspot / compilerDiscussionhotspot dash compiler dash dev at openjdk dot java dot netEffortLDurationMReviewed byMikael VidstedtEndorsed byMikael VidstedtCreated2014/05/16 08:58Updated2017/04/28 06:26Issue8043304Summary
Divide the code cache into distinct segments, each of which contains
compiled code of a particular type, in order to improve performance and
enable future extensions.
Goals

Separate non-method, profiled, and non-profiled code
Shorter sweep times due to specialized iterators that skip non-method code
Improve execution time for some compilation-intensive benchmarks
Better control of JVM memory footprint
Decrease fragmentation of highly-optimized code
Improve code locality because code of the same type is likely to be
accessed close in time

Better iTLB and iCache behavior


Establish a base for future extensions

Improved management of heterogeneous code; for example, Sumatra
(GPU code) and AOT compiled code
Possibility of fine-grained locking per code heap
Future separation of code and metadata (see
JDK-7072317)



Non-Goals
The segmented code cache only provides a base for future extensions such
as fine-grained locking; it does not yet implement any of these
improvements.
Success Metrics

Separation of different code types
Shorter sweep time
Lower execution time
Decreased fragmentation of highly optimized code
Reduced number of iTLB and iCache misses

Motivation
The organization and maintenance of compiled code has a significant
impact on performance. Instances of performance regressions of several factors have been reported if the code cache
takes the wrong actions. With the introduction of tiered compilation the
role of the code cache has become even more important, since the amount of
compiled code increases by a factor of 2X--4X compared to using
non-tiered compilation. Tiered compilation also introduces a new compiled
code type: instrumented compiled code (profiled code). Profiled code has
different properties than non-profiled code; one important difference is
that profiled code has a predefined, limited lifetime while non-profiled
code potentially remains in the code cache forever.
The current code cache is optimized to handle homogeneous code, i.e.,
only one type of compiled code. The code cache is organized as a single
heap data structure on top of a contiguous chunk of memory. Therefore,
profiled code which has a predefined limited lifetime is mixed with
non-profiled code, which potentially remains in the code cache
forever. This leads to different performance and design problems. For
example, the method sweeper has to scan the entire code cache while
sweeping, even if some entries are never flushed or contain non-method
code.
Description
Instead of having a single code heap, the code cache is segmented into
distinct code heaps, each of which contains compiled code of a particular
type. Such a design enables us to separate code with different
properties. There are three different top-level types of compiled code:

JVM internal (non-method) code
Profiled-code
Non-profiled code

The corresponding code heaps are:


A non-method code heap containing non-method code, such as compiler
buffers and bytecode interpreter. This code type will stay in the
code cache forever.


A profiled code heap containing lightly optimized, profiled methods
with a short lifetime.


A non-profiled code heap containing fully optimized, non-profiled
methods with a potentially long lifetime.


The non-method code heap has a fixed size of 3 MB to account for the VM
internals plus additional space for the compiler buffers. This additional
space is adjusted according to the number of C1/C2 compiler threads. The
remaining code cache space is distributed evenly among the profiled and
the non-profiled code heaps.
The following command-line switches are introduced to control the sizes
of the code heaps:


-XX:NonProfiledCodeHeapSize: Sets the size in bytes of the code
heap containing non-profiled methods.


-XX:ProfiledCodeHeapSize: Sets the size in bytes of the code heap
containing profiled methods.


-XX:NonMethodCodeHeapSize: Sets the size in bytes of the code heap
containing non-method code.


The interface and implementation of the code cache is adapted to support
multiple code heaps. Because the code cache is a central component of the
JVM, many other components are affected by these changes, including the
following:

Code cache sweeper: Now only iterates over the method-code heaps
Tiered compilation policy: Sets compilation thresholds according to
free space in code heaps
Java Flight Recorder (JFR): Events related to the code cache
Indirect references from:

Serviceability Agent: Java interface to code-cache internals
DTrace ustack helper script (jhelper.d): Resolving of names of
compiled Java methods
Pstack support library (libjvm_db.c): Stack tracing of compiled
Java methods



Alternatives
An alternative implementation would define logical memory regions into
which different code types are preferably allocated. If there is free
space, we allocate into the preferred memory region and if there is no
free space left, we allocate somewhere else.
Testing
Intensive correctness testing using JPRT, Nashorn + Octane, SPECjbb2013,
SPECjbb2005, SPECjvm2008.
We need to make sure that there is no performance degredation, especially
for embedded usage with small code-cache sizes.
Testing of affected components including Serviceability Agent, DTrace,
Pstack, Java Flight Recorder.
Risks and Assumptions
Having a fixed size per code heap leads to a potential waste of memory in
case one code heap is full and there is still space in another code
heap. Especially for very small code cache sizes it may happen that the
compilers are shut off even if there is still space available. To solve
this problem an option will be added to turn off the segmentation for
small code-cache sizes.
The size of the non-method code depends on the Java application, the
underlying platform, and the JVM settings. It is therefore hard to
determine the required space in the non-method code heap at JVM startup.
Future versions of this patch may implement dynamic resizing (supported
by the sweeper) or different allocation strategies to lower the risk of
wasting memory.
InstallingContributingSponsoringDevelopers' GuideVulnerabilitiesJDK GA/EA BuildsMailing listsWiki
                      · IRCBylaws · CensusLegalWorkshopJEP ProcessSource codeMercurialGitHubToolsGitjtreg harnessGroups(overview)AdoptionBuildClient LibrariesCompatibility & Specification ReviewCompilerConformanceCore LibrariesGoverning BoardHotSpotIDE Tooling & SupportInternationalizationJMXMembersNetworkingPortersQualitySecurityServiceabilityVulnerabilityWebProjects(overview,
      archive)AmberBabylonCRaCCaciocavalloClosuresCode ToolsCoinCommon VM InterfaceCompiler GrammarDetroitDevelopers' GuideDevice I/ODukeGalahadGraalIcedTeaJDK 7JDK 8JDK 8 UpdatesJDK 9JDK
      (…,
       21,
       22,
       23)JDK UpdatesJavaDoc.NextJigsawKonaKullaLambdaLanaiLeydenLilliputLocale EnhancementLoomMemory Model UpdateMetropolisMission ControlMulti-Language VMNashornNew I/OOpenJFXPanamaPenrosePort: AArch32Port: AArch64Port: BSDPort: HaikuPort: Mac OS XPort: MIPSPort: MobilePort: PowerPC/AIXPort: RISC-VPort: s390xPortolaSCTPShenandoahSkaraSumatraTiered AttributionTsanType AnnotationsValhallaVeronaVisualVMWakefieldZeroZGC

        © 2024 Oracle Corporation and/or its affiliates
        Terms of Use
        ·
        
            License: GPLv2
        · Privacy
        · Trademarks
