JEP 233: Generate Run-Time Compiler Tests AutomaticallyJEP 233: Generate Run-Time Compiler Tests AutomaticallyOwnerIgor IgnatyevTypeFeatureScopeImplementationStatusClosed / DeliveredRelease9Componenthotspot / compilerDiscussionhotspot dash compiler dash dev at openjdk dot java dot netEffortLDurationMReviewed byAleksandre Iline, Igor Veresov, Mikael Vidstedt, Vladimir KozlovEndorsed byMikael VidstedtCreated2014/05/19 17:57Updated2017/04/10 04:25Issue8043467Summary
Develop a tool to test the run-time compilers by automatically generating
test cases.
Goals


Generated tests should be jtreg-compatible


The tool should be configurable in terms of the language of final
result (Java source code, Java bytecode), use of language constructs,
control-flow and expressions complexity, etc.


Tests should be generated in randomly but reproducibly


Motivation
As we add new platforms, leverage new CPU instructions, introduce new
optimizations, and make other enhancements to the run-time compilers, it
becomes increasingly infeasible to test the compilers effectively with
direct, targeted tests.
Description
The tool will randomly generate syntactically and semantically correct
Java source code or byte code, compile it if necessary, run it in both
interpreted (-Xint) and compiled (-Xcomp) modes, and verify the
results.
The tool will work automatically, without human interaction.  The
generated tests will cover as many combinations as possible in a
reasonable amount of time.
The Java source-code compiler, javac, does not use all of Java's byte
codes, so generating only Java source code would leave some byte codes
uncovered.  Generating only byte code for all types of tests would be a
much more complicated task, so we will adopt a hybrid approach that
generates both Java source code and byte code.
Compiling source code during test execution is problematic for embedded
platforms, where a full JDK might not be available, so the tool will
provide a way to pre-compile source-code tests.
The generated test cases will include complicated expressions and
control-flow graphs and will make use of intrinsics, floating-point
operations, try-catch-finally constructs, etc.  There will be a way to
adjust the tool's configuration.
The tool will generate tests randomly, but for reproducibility it will
report its randomization seed and accept such a seed in order to replay
the exact same sequence of generated tests.
The tool's source code will be placed in the
hotspot/test/testlibrary/jit-tester directory.  Tests can be generated
via targets provided in the tool's makefile.  The result of test
generation is a complete jtreg test suite which can be run from the
same makefile or via jtreg directly.  The tool makefiles will not be
integrated into the HotSpot/JDK build infrastructure.
Given that the test generation process takes a significant amount of
time, generating and running these tests is not expected to be a part of
pre-integration testing.  It makes sense, however, to regularly run
pre-generated tests, for reliability testing, and new generated tests, to
get better code coverage.  Generated tests which find bugs should be
integrated as regular regression tests into an appropriate test suite and
run in the same way as other regression tests.
Alternatives
Running existing tests in compiled modes can be considered as a viable
alternative for the tool.  Such an approach has several drawbacks:


It doesn't guarantee covering all language constructs and
combinations of different optimization


Test failure doesn't always mean a defect in run-time compilers,
additional engineer's time for investigation/reproduction is needed


Since some of tests can be run-time compiler specific tests, they may
force run-time compilation themselves and/or require specific
run-time compiler states.  So explicit compiled mode can change the
tests behavior and lead to false positive results


Creating a regression test for test failure is relatively harder


Due to these drawbacks this approach can not fully replace the proposed
tool.
InstallingContributingSponsoringDevelopers' GuideVulnerabilitiesJDK GA/EA BuildsMailing listsWiki
                      · IRCBylaws · CensusLegalWorkshopJEP ProcessSource codeMercurialGitHubToolsGitjtreg harnessGroups(overview)AdoptionBuildClient LibrariesCompatibility & Specification ReviewCompilerConformanceCore LibrariesGoverning BoardHotSpotIDE Tooling & SupportInternationalizationJMXMembersNetworkingPortersQualitySecurityServiceabilityVulnerabilityWebProjects(overview,
      archive)AmberBabylonCRaCCaciocavalloClosuresCode ToolsCoinCommon VM InterfaceCompiler GrammarDetroitDevelopers' GuideDevice I/ODukeGalahadGraalIcedTeaJDK 7JDK 8JDK 8 UpdatesJDK 9JDK
      (…,
       21,
       22,
       23)JDK UpdatesJavaDoc.NextJigsawKonaKullaLambdaLanaiLeydenLilliputLocale EnhancementLoomMemory Model UpdateMetropolisMission ControlMulti-Language VMNashornNew I/OOpenJFXPanamaPenrosePort: AArch32Port: AArch64Port: BSDPort: HaikuPort: Mac OS XPort: MIPSPort: MobilePort: PowerPC/AIXPort: RISC-VPort: s390xPortolaSCTPShenandoahSkaraSumatraTiered AttributionTsanType AnnotationsValhallaVeronaVisualVMWakefieldZeroZGC

        © 2024 Oracle Corporation and/or its affiliates
        Terms of Use
        ·
        
            License: GPLv2
        · Privacy
        · Trademarks
