source: webkit/trunk/Source/JavaScriptCore/dfg/DFGCFAPhase.cpp

Last change on this file was 280760, checked in by keith_miller@apple.com, 4 years ago

for-in should only emit one loop in bytecode
https://bugs.webkit.org/show_bug.cgi?id=227989

Reviewed by Yusuke Suzuki.

JSTests:

  • microbenchmarks/for-in-double-array-with-own-named.js: Added.

(test):

  • microbenchmarks/for-in-double-array.js: Added.

(test):

  • microbenchmarks/for-in-getters.js: Added.

(test):

  • microbenchmarks/for-in-int32-array-with-own-named.js: Added.

(test):

  • microbenchmarks/for-in-int32-array.js: Added.

(test):

  • microbenchmarks/for-in-int32-object-with-own-named-and-getters.js: Added.

(test):

  • microbenchmarks/for-in-int32-object-with-own-named.js: Added.

(test):

  • microbenchmarks/for-in-object-with-own-named.js: Added.

(sum):
(opaqueSet):

  • microbenchmarks/for-in-string-array.js: Added.

(test):

  • microbenchmarks/for-of-iterate-array-map-set.js: Added.

(sum):
(let.generator):

  • stress/for-in-array-mode.js:

(test):

  • stress/for-in-base-reassigned-later.js:
  • stress/for-in-delete-during-iteration.js:
  • stress/for-in-primitive-index-on-prototype.js: Added.

(test):

  • stress/for-in-tests.js:
  • stress/has-own-property-structure-for-in-loop-correctness.js:

(test5):

Source/JavaScriptCore:

This patch redesigns how we implement for-in loops. Before this patch we would emit three copies of the for-in loop body. One for the indexed properties, one for the named-own properties, and one for generic properties (anything else). This had a couple of problems. Firstly, it meant bytecode size grew exponentially to number of nested for-in loops. This in turn meant DFG/FTL compilation took much longer.

Going off our experience with fast for-of, this patch turns for-in loops specializations into
a "fused" opcode that internally switches on the enumeration mode it currently sees. For example, if we are enumerating an own-named property, the new enumerator_get_by_val bytecode will check the enumerator cell's cached structure matches the base's then load the property offset directly.

There are four new opcodes this patch adds, which replace the various operations we had for the specialized loops previously. The new opcodes are EnumeratorGetByVal, EnumeratorInByVal, EnumeratorHasOwnProperty, and EnumeratorNext. The first three correspond to GetByVal, InByVal, and HasOwnProperty respectively. The EnumeratorNext opcode has three results in bytecode, the next enumeration value's mode, the index of the property name, and the property name string itself. When enumeration is done EnumeratorNext returns JS null as the property name string. Since the DFG doesn't support tuples yet this opcode is spilt into four new nodes. The first computes the updated index and mode for the next enumeration key, which is encoded into a single JS number. Then there are two nodes that extract the mode and index. Finally, the last new node produces the property name string or null based on the extracted mode and index.

Since, in most benchmarks, any given enumeration opcode tends to profile exactly one enumeration mode. This patch focuses primarily on reimplementing all the optimizations we have for any one specific mode. This means there are still potential optimizations for the multi-mode flavors of each new opcode.

The main optimizations implemented for each new opcode are:

EnumeratorNext:
1) IndexedMode loops are loaded and checked for presence inline (DFG/FTL).
2) NamedMode is computed inline as long as the cached structure on the enumerator cell matches the base (Baseline+). This can only differ if there's a transition.
3) property names are extracted from the cached buffer inline (Baseline+).

EnumeratorGetByVal:
EnumeratorInByVal:
EnumeratorHasOwnProperty:
1) IndexedMode has all the optimizations of a normal XByVal on indexed properties (DFG/FTL).
2) NamedMode will extract the value directly from the inline/out-of-line offset if the structure matches the enumerator's (Baseline+).

There are also a few interesting changes worth mentioning here:
1) If a for-in loop would produce an empty enumerator we now always
return the VMs empty enumerator. This has two benefits, most importantly, it distingishes between an unprofiled for-in loop and empty enumeration, which prevents OSR exit loops. Also, it means that the various Enumerator opcodes no longer need to handle undefined/null when toObjecting the base value.

2) The enumerator now contains a bit set of all the modes it will produce. This removes a few extra branches when speculating on the modes we will see in EnumeratorNext.

3) In the DFG, enumerator GetByVal relies on compileGetByVal to set the result it also passes a prefix callback which emits code after the various cases set up their operands but before code is emitting to help satisfy the branch over register allocation validation. Also, the array mode branch in compileGetByVal passes the data format that it would prefer, which for normal GetByVal is returned. For EnumeratorGetByVal, that preference is completely ignored and it always returns DataFormatJS.

  • assembler/MacroAssemblerARM64.h:

(JSC::MacroAssemblerARM64::or8):

  • assembler/MacroAssemblerX86Common.h:

(JSC::MacroAssemblerX86Common::or8):

  • assembler/MacroAssemblerX86_64.h:

(JSC::MacroAssemblerX86_64::rshift64):
(JSC::MacroAssemblerX86_64::or8): Deleted.

  • builtins/BuiltinNames.h:
  • bytecode/BytecodeList.rb:
  • bytecode/BytecodeUseDef.cpp:

(JSC::computeUsesForBytecodeIndexImpl):
(JSC::computeDefsForBytecodeIndexImpl):

  • bytecode/CodeBlock.cpp:

(JSC::CodeBlock::finishCreation):

  • bytecode/LinkTimeConstant.h:
  • bytecode/Opcode.h:
  • bytecompiler/BytecodeGenerator.cpp:

(JSC::BytecodeGenerator::recordHasOwnPropertyInForInLoop):
(JSC::BytecodeGenerator::emitInByVal):
(JSC::BytecodeGenerator::emitGetByVal):
(JSC::BytecodeGenerator::emitEnumeratorNext):
(JSC::BytecodeGenerator::emitEnumeratorHasOwnProperty):
(JSC::BytecodeGenerator::pushForInScope):
(JSC::BytecodeGenerator::popForInScope):
(JSC::rewriteOp):
(JSC::ForInContext::finalize):
(JSC::BytecodeGenerator::findForInContext):
(JSC::BytecodeGenerator::recordHasOwnStructurePropertyInForInLoop): Deleted.
(JSC::BytecodeGenerator::emitGetEnumerableLength): Deleted.
(JSC::BytecodeGenerator::emitHasEnumerableIndexedProperty): Deleted.
(JSC::BytecodeGenerator::emitHasEnumerableStructureProperty): Deleted.
(JSC::BytecodeGenerator::emitHasEnumerableProperty): Deleted.
(JSC::BytecodeGenerator::emitHasOwnStructureProperty): Deleted.
(JSC::BytecodeGenerator::emitEnumeratorStructurePropertyName): Deleted.
(JSC::BytecodeGenerator::emitEnumeratorGenericPropertyName): Deleted.
(JSC::BytecodeGenerator::emitToIndexString): Deleted.
(JSC::BytecodeGenerator::pushIndexedForInScope): Deleted.
(JSC::BytecodeGenerator::popIndexedForInScope): Deleted.
(JSC::BytecodeGenerator::pushStructureForInScope): Deleted.
(JSC::BytecodeGenerator::popStructureForInScope): Deleted.
(JSC::StructureForInContext::finalize): Deleted.
(JSC::IndexedForInContext::finalize): Deleted.
(JSC::BytecodeGenerator::findStructureForInContext): Deleted.

  • bytecompiler/BytecodeGenerator.h:

(JSC::ForInContext::isValid const):
(JSC::ForInContext::invalidate):
(JSC::ForInContext::local const):
(JSC::ForInContext::propertyName const):
(JSC::ForInContext::propertyOffset const):
(JSC::ForInContext::enumerator const):
(JSC::ForInContext::mode const):
(JSC::ForInContext::ForInContext):
(JSC::ForInContext::bodyBytecodeStartOffset const):
(JSC::ForInContext::type const): Deleted.
(JSC::ForInContext::isIndexedForInContext const): Deleted.
(JSC::ForInContext::isStructureForInContext const): Deleted.
(JSC::ForInContext::asIndexedForInContext): Deleted.
(JSC::ForInContext::asStructureForInContext): Deleted.
(JSC::StructureForInContext::StructureForInContext): Deleted.
(JSC::StructureForInContext::index const): Deleted.
(JSC::StructureForInContext::property const): Deleted.
(JSC::StructureForInContext::enumerator const): Deleted.
(JSC::StructureForInContext::baseVariable const): Deleted.
(JSC::StructureForInContext::addGetInst): Deleted.
(JSC::StructureForInContext::addInInst): Deleted.
(JSC::StructureForInContext::addHasOwnPropertyJump): Deleted.
(JSC::IndexedForInContext::IndexedForInContext): Deleted.
(JSC::IndexedForInContext::index const): Deleted.
(JSC::IndexedForInContext::addGetInst): Deleted.

  • bytecompiler/NodesCodegen.cpp:

(JSC::HasOwnPropertyFunctionCallDotNode::emitBytecode):
(JSC::ForInNode::emitBytecode):

  • dfg/DFGAbstractInterpreterInlines.h:

(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):

  • dfg/DFGArrayMode.h:

(JSC::DFG::ArrayMode::isSaneChain const):

  • dfg/DFGBackwardsPropagationPhase.cpp:

(JSC::DFG::BackwardsPropagationPhase::propagate):

  • dfg/DFGByteCodeParser.cpp:

(JSC::DFG::ByteCodeParser::parseBlock):

  • dfg/DFGCFAPhase.cpp:

(JSC::DFG::CFAPhase::injectOSR):

  • dfg/DFGCapabilities.cpp:

(JSC::DFG::capabilityLevel):

  • dfg/DFGClobberize.h:

(JSC::DFG::clobberize):

  • dfg/DFGDoesGC.cpp:

(JSC::DFG::doesGC):

  • dfg/DFGFixupPhase.cpp:

(JSC::DFG::FixupPhase::fixupNode):
(JSC::DFG::FixupPhase::setJSArraySaneChainIfPossible):

  • dfg/DFGGraph.cpp:

(JSC::DFG::Graph::dump):

  • dfg/DFGIntegerRangeOptimizationPhase.cpp:
  • dfg/DFGMayExit.cpp:
  • dfg/DFGNode.h:

(JSC::DFG::Node::hasHeapPrediction):
(JSC::DFG::Node::hasStorageChild const):
(JSC::DFG::Node::storageChildIndex):
(JSC::DFG::Node::hasArrayMode):
(JSC::DFG::Node::hasEnumeratorMetadata const):
(JSC::DFG::Node::enumeratorMetadata):

  • dfg/DFGNodeType.h:
  • dfg/DFGOpInfo.h:

(JSC::DFG::OpInfo::OpInfo):

  • dfg/DFGOperations.cpp:

(JSC::DFG::JSC_DEFINE_JIT_OPERATION):

  • dfg/DFGOperations.h:
  • dfg/DFGPredictionPropagationPhase.cpp:
  • dfg/DFGSSALoweringPhase.cpp:

(JSC::DFG::SSALoweringPhase::handleNode):

  • dfg/DFGSafeToExecute.h:

(JSC::DFG::safeToExecute):

  • dfg/DFGSpeculativeJIT.cpp:

(JSC::DFG::JSValueRegsTemporary::JSValueRegsTemporary):
(JSC::DFG::SpeculativeJIT::compileGetByValOnString):
(JSC::DFG::SpeculativeJIT::setIntTypedArrayLoadResult):
(JSC::DFG::SpeculativeJIT::compileGetByValOnIntTypedArray):
(JSC::DFG::SpeculativeJIT::compileGetByValOnFloatTypedArray):
(JSC::DFG::SpeculativeJIT::compileGetByValForObjectWithString):
(JSC::DFG::SpeculativeJIT::compileGetByValForObjectWithSymbol):
(JSC::DFG::SpeculativeJIT::compileGetByValOnDirectArguments):
(JSC::DFG::SpeculativeJIT::compileGetByValOnScopedArguments):
(JSC::DFG::SpeculativeJIT::compileEnumeratorNextUpdateIndexAndMode):
(JSC::DFG::SpeculativeJIT::compileEnumeratorNextExtractIndex):
(JSC::DFG::SpeculativeJIT::compileEnumeratorNextExtractMode):
(JSC::DFG::SpeculativeJIT::compileEnumeratorNextUpdatePropertyName):
(JSC::DFG::SpeculativeJIT::compileEnumeratorGetByVal):
(JSC::DFG::SpeculativeJIT::compileEnumeratorHasProperty):
(JSC::DFG::SpeculativeJIT::compileEnumeratorInByVal):
(JSC::DFG::SpeculativeJIT::compileEnumeratorHasOwnProperty):
(JSC::DFG::SpeculativeJIT::compileHasIndexedProperty):
(JSC::DFG::SpeculativeJIT::compileGetEnumerableLength): Deleted.
(JSC::DFG::SpeculativeJIT::compileHasEnumerableProperty): Deleted.
(JSC::DFG::SpeculativeJIT::compileToIndexString): Deleted.
(JSC::DFG::SpeculativeJIT::compileHasEnumerableStructureProperty): Deleted.
(JSC::DFG::SpeculativeJIT::compileHasOwnStructurePropertyImpl): Deleted.
(JSC::DFG::SpeculativeJIT::compileHasOwnStructureProperty): Deleted.
(JSC::DFG::SpeculativeJIT::compileInStructureProperty): Deleted.
(JSC::DFG::SpeculativeJIT::compileGetEnumeratorPname): Deleted.
(JSC::DFG::SpeculativeJIT::compileGetDirectPname): Deleted.

  • dfg/DFGSpeculativeJIT.h:

(JSC::DFG::SpeculativeJIT::allocate):
(JSC::DFG::JSValueOperand::regs):
(JSC::DFG::JSValueOperand::gpr):
(JSC::DFG::StorageOperand::StorageOperand):
(JSC::DFG::StorageOperand::~StorageOperand):
(JSC::DFG::StorageOperand::emplace):
(JSC::DFG::JSValueRegsTemporary::operator bool):
(JSC::DFG::JSValueRegsTemporary::JSValueRegsTemporary):

  • dfg/DFGSpeculativeJIT32_64.cpp:

(JSC::DFG::SpeculativeJIT::compileGetByVal):
(JSC::DFG::SpeculativeJIT::compile):

  • dfg/DFGSpeculativeJIT64.cpp:

(JSC::DFG::SpeculativeJIT::compileGetByVal):
(JSC::DFG::SpeculativeJIT::compile):

  • dfg/DFGTypeCheckHoistingPhase.cpp:

(JSC::DFG::TypeCheckHoistingPhase::identifyRedundantStructureChecks):
(JSC::DFG::TypeCheckHoistingPhase::identifyRedundantArrayChecks):

  • ftl/FTLAbstractHeapRepository.h:
  • ftl/FTLCapabilities.cpp:

(JSC::FTL::canCompile):

  • ftl/FTLLowerDFGToB3.cpp:

(JSC::FTL::DFG::LowerDFGToB3::compileNode):
(JSC::FTL::DFG::LowerDFGToB3::compileGetByValImpl):
(JSC::FTL::DFG::LowerDFGToB3::compileGetByVal):
(JSC::FTL::DFG::LowerDFGToB3::compileStringCharAtImpl):
(JSC::FTL::DFG::LowerDFGToB3::compileStringCharAt):
(JSC::FTL::DFG::LowerDFGToB3::compileCompareStrictEq):

  • ftl/FTLOutput.h:

(JSC::FTL::Output::phi):

  • generator/DSL.rb:
  • interpreter/Register.h:
  • interpreter/RegisterInlines.h:

(JSC::Register::operator=):

  • jit/AssemblyHelpers.h:
  • jit/JIT.cpp:

(JSC::JIT::privateCompileMainPass):
(JSC::JIT::privateCompileSlowCases):

  • jit/JIT.h:
  • jit/JITOpcodes.cpp:

(JSC::JIT::privateCompileHasIndexedProperty):
(JSC::JIT::emit_op_has_structure_propertyImpl): Deleted.
(JSC::JIT::emit_op_has_enumerable_structure_property): Deleted.
(JSC::JIT::emit_op_has_own_structure_property): Deleted.
(JSC::JIT::emit_op_in_structure_property): Deleted.
(JSC::JIT::emit_op_has_enumerable_indexed_property): Deleted.
(JSC::JIT::emitSlow_op_has_enumerable_indexed_property): Deleted.
(JSC::JIT::emit_op_get_direct_pname): Deleted.
(JSC::JIT::emit_op_enumerator_structure_pname): Deleted.
(JSC::JIT::emit_op_enumerator_generic_pname): Deleted.

  • jit/JITOpcodes32_64.cpp:

(JSC::JIT::privateCompileHasIndexedProperty):
(JSC::JIT::emit_op_has_structure_propertyImpl): Deleted.
(JSC::JIT::emit_op_has_enumerable_structure_property): Deleted.
(JSC::JIT::emit_op_has_own_structure_property): Deleted.
(JSC::JIT::emit_op_in_structure_property): Deleted.
(JSC::JIT::emit_op_has_enumerable_indexed_property): Deleted.
(JSC::JIT::emitSlow_op_has_enumerable_indexed_property): Deleted.
(JSC::JIT::emit_op_get_direct_pname): Deleted.
(JSC::JIT::emit_op_enumerator_structure_pname): Deleted.
(JSC::JIT::emit_op_enumerator_generic_pname): Deleted.

  • jit/JITPropertyAccess.cpp:

(JSC::JIT::generateGetByValSlowCase):
(JSC::JIT::emitSlow_op_get_by_val):
(JSC::JIT::emit_op_enumerator_next):
(JSC::JIT::emit_op_enumerator_get_by_val):
(JSC::JIT::emitSlow_op_enumerator_get_by_val):
(JSC::JIT::emit_enumerator_has_propertyImpl):
(JSC::JIT::emit_op_enumerator_in_by_val):
(JSC::JIT::emit_op_enumerator_has_own_property):

  • jit/JITPropertyAccess32_64.cpp:

(JSC::JIT::emit_op_enumerator_next):
(JSC::JIT::emit_op_enumerator_get_by_val):
(JSC::JIT::emitSlow_op_enumerator_get_by_val):
(JSC::JIT::emit_op_enumerator_in_by_val):
(JSC::JIT::emit_op_enumerator_has_own_property):

  • llint/LowLevelInterpreter.asm:
  • llint/LowLevelInterpreter64.asm:
  • runtime/CommonSlowPaths.cpp:

(JSC::JSC_DEFINE_COMMON_SLOW_PATH):

  • runtime/CommonSlowPaths.h:
  • runtime/FileBasedFuzzerAgent.cpp:

(JSC::FileBasedFuzzerAgent::getPredictionInternal):

  • runtime/FileBasedFuzzerAgentBase.cpp:

(JSC::FileBasedFuzzerAgentBase::opcodeAliasForLookupKey):

  • runtime/JSGlobalObject.cpp:

(JSC::JSGlobalObject::init):

  • runtime/JSPropertyNameEnumerator.cpp:

(JSC::JSPropertyNameEnumerator::JSPropertyNameEnumerator):
(JSC::JSPropertyNameEnumerator::computeNext):

  • runtime/JSPropertyNameEnumerator.h:

(JSC::propertyNameEnumerator):

  • runtime/PredictionFileCreatingFuzzerAgent.cpp:

(JSC::PredictionFileCreatingFuzzerAgent::getPredictionInternal):

File size: 11.7 KB
Line 
1/*
2 * Copyright (C) 2011-2018 Apple Inc. All rights reserved.
3 *
4 * Redistribution and use in source and binary forms, with or without
5 * modification, are permitted provided that the following conditions
6 * are met:
7 * 1. Redistributions of source code must retain the above copyright
8 * notice, this list of conditions and the following disclaimer.
9 * 2. Redistributions in binary form must reproduce the above copyright
10 * notice, this list of conditions and the following disclaimer in the
11 * documentation and/or other materials provided with the distribution.
12 *
13 * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
14 * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
15 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
16 * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
17 * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
18 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
19 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
20 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
21 * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
22 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
23 * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
24 */
25
26#include "config.h"
27#include "DFGCFAPhase.h"
28
29#if ENABLE(DFG_JIT)
30
31#include "DFGAbstractInterpreterInlines.h"
32#include "DFGBlockSet.h"
33#include "DFGClobberSet.h"
34#include "DFGClobberize.h"
35#include "DFGGraph.h"
36#include "DFGInPlaceAbstractState.h"
37#include "DFGPhase.h"
38#include "DFGSafeToExecute.h"
39#include "OperandsInlines.h"
40#include "JSCInlines.h"
41
42namespace JSC { namespace DFG {
43
44class CFAPhase : public Phase {
45public:
46 CFAPhase(Graph& graph)
47 : Phase(graph, "control flow analysis")
48 , m_state(graph)
49 , m_interpreter(graph, m_state)
50 , m_verbose(Options::verboseCFA())
51 {
52 }
53
54 bool run()
55 {
56 ASSERT(m_graph.m_form == ThreadedCPS || m_graph.m_form == SSA);
57 ASSERT(m_graph.m_unificationState == GloballyUnified);
58 ASSERT(m_graph.m_refCountState == EverythingIsLive);
59
60 m_count = 0;
61
62 if (m_verbose && !shouldDumpGraphAtEachPhase(m_graph.m_plan.mode())) {
63 dataLog("Graph before CFA:\n");
64 m_graph.dump();
65 }
66
67 // This implements a pseudo-worklist-based forward CFA, except that the visit order
68 // of blocks is the bytecode program order (which is nearly topological), and
69 // instead of a worklist we just walk all basic blocks checking if cfaShouldRevisit
70 // is set to true. This is likely to balance the efficiency properties of both
71 // worklist-based and forward fixpoint-based approaches. Like a worklist-based
72 // approach, it won't visit code if it's meaningless to do so (nothing changed at
73 // the head of the block or the predecessors have not been visited). Like a forward
74 // fixpoint-based approach, it has a high probability of only visiting a block
75 // after all predecessors have been visited. Only loops will cause this analysis to
76 // revisit blocks, and the amount of revisiting is proportional to loop depth.
77
78 m_state.initialize();
79
80 if (m_graph.m_form != SSA) {
81 if (m_verbose)
82 dataLog(" Widening state at OSR entry block.\n");
83
84 // Widen the abstract values at the block that serves as the must-handle OSR entry.
85 for (BlockIndex blockIndex = m_graph.numBlocks(); blockIndex--;) {
86 BasicBlock* block = m_graph.block(blockIndex);
87 if (!block)
88 continue;
89
90 if (!block->isOSRTarget)
91 continue;
92 if (block->bytecodeBegin != m_graph.m_plan.osrEntryBytecodeIndex())
93 continue;
94
95 // We record that the block needs some OSR stuff, but we don't do that yet. We want to
96 // handle OSR entry data at the right time in order to get the best compile times. If we
97 // simply injected OSR data right now, then we'd potentially cause a loop body to be
98 // interpreted with just the constants we feed it, which is more expensive than if we
99 // interpreted it with non-constant values. If we always injected this data after the
100 // main pass of CFA ran, then we would potentially spend a bunch of time rerunning CFA
101 // after convergence. So, we try very hard to inject OSR data for a block when we first
102 // naturally come to see it - see the m_blocksWithOSR check in performBlockCFA(). This
103 // way, we:
104 //
105 // - Reduce the likelihood of interpreting the block with constants, since we will inject
106 // the OSR entry constants on top of whatever abstract values we got for that block on
107 // the first pass. The mix of those two things is likely to not be constant.
108 //
109 // - Reduce the total number of CFA reexecutions since we inject the OSR data as part of
110 // the normal flow of CFA instead of having to do a second fixpoint. We may still have
111 // to do a second fixpoint if we don't even reach the OSR entry block during the main
112 // run of CFA, but in that case at least we're not being redundant.
113 m_blocksWithOSR.add(block);
114 }
115 }
116
117 do {
118 m_changed = false;
119 performForwardCFA();
120 } while (m_changed);
121
122 if (m_graph.m_form != SSA) {
123 for (BlockIndex blockIndex = m_graph.numBlocks(); blockIndex--;) {
124 BasicBlock* block = m_graph.block(blockIndex);
125 if (!block)
126 continue;
127
128 if (m_blocksWithOSR.remove(block))
129 m_changed |= injectOSR(block);
130 }
131
132 while (m_changed) {
133 m_changed = false;
134 performForwardCFA();
135 }
136
137 // Make sure we record the intersection of all proofs that we ever allowed the
138 // compiler to rely upon.
139 for (BlockIndex blockIndex = m_graph.numBlocks(); blockIndex--;) {
140 BasicBlock* block = m_graph.block(blockIndex);
141 if (!block)
142 continue;
143
144 block->intersectionOfCFAHasVisited &= block->cfaHasVisited;
145 for (unsigned i = block->intersectionOfPastValuesAtHead.size(); i--;) {
146 AbstractValue value = block->valuesAtHead[i];
147 // We need to guarantee that when we do an OSR entry, we validate the incoming
148 // value as if it could be live past an invalidation point. Otherwise, we may
149 // OSR enter with a value with the wrong structure, and an InvalidationPoint's
150 // promise of filtering the structure set of certain values is no longer upheld.
151 value.m_structure.observeInvalidationPoint();
152 block->intersectionOfPastValuesAtHead[i].filter(value);
153 }
154 }
155 }
156
157 return true;
158 }
159
160private:
161 bool injectOSR(BasicBlock* block)
162 {
163 if (m_verbose)
164 dataLog(" Found must-handle block: ", *block, "\n");
165
166 // This merges snapshot of stack values while CFA phase want to have proven types and values. This is somewhat tricky.
167 // But this is OK as long as DFG OSR entry validates the inputs with *proven* AbstractValue values. And it turns out that this
168 // type widening is critical to navier-stokes. Without it, navier-stokes has more strict constraint on OSR entry and
169 // fails OSR entry repeatedly.
170 bool changed = false;
171 const Operands<std::optional<JSValue>>& mustHandleValues = m_graph.m_plan.mustHandleValues();
172 for (size_t i = mustHandleValues.size(); i--;) {
173 Operand operand = mustHandleValues.operandForIndex(i);
174 std::optional<JSValue> value = mustHandleValues[i];
175 if (!value) {
176 if (m_verbose)
177 dataLog(" Not live in bytecode: ", operand, "\n");
178 continue;
179 }
180 Node* node = block->variablesAtHead.operand(operand);
181 if (!node) {
182 if (m_verbose)
183 dataLog(" Not live: ", operand, "\n");
184 continue;
185 }
186
187 if (m_verbose)
188 dataLog(" Widening ", operand, " with ", value.value(), "\n");
189
190 AbstractValue& target = block->valuesAtHead.operand(operand);
191 changed |= target.mergeOSREntryValue(m_graph, value.value(), node->variableAccessData(), node);
192 }
193
194 if (changed || !block->cfaHasVisited) {
195 block->cfaShouldRevisit = true;
196 return true;
197 }
198
199 return false;
200 }
201
202 void performBlockCFA(BasicBlock* block)
203 {
204 if (!block)
205 return;
206 if (!block->cfaShouldRevisit)
207 return;
208 if (m_verbose)
209 dataLog(" Block ", *block, ":\n");
210
211 if (m_blocksWithOSR.remove(block))
212 injectOSR(block);
213
214 m_state.beginBasicBlock(block);
215 if (m_verbose) {
216 dataLog(" head vars: ", block->valuesAtHead, "\n");
217 if (m_graph.m_form == SSA)
218 dataLog(" head regs: ", nodeValuePairListDump(block->ssa->valuesAtHead), "\n");
219 }
220 for (unsigned i = 0; i < block->size(); ++i) {
221 Node* node = block->at(i);
222 if (m_verbose) {
223 dataLogF(" %s @%u: ", Graph::opName(node->op()), node->index());
224
225 if (!safeToExecute(m_state, m_graph, node))
226 dataLog("(UNSAFE) ");
227
228 dataLog(m_state.variablesForDebugging(), " ", m_interpreter);
229
230 dataLogF("\n");
231 }
232 if (!m_interpreter.execute(i)) {
233 if (m_verbose)
234 dataLogF(" Expect OSR exit.\n");
235 break;
236 }
237
238 if (ASSERT_ENABLED
239 && m_state.didClobberOrFolded() != writesOverlap(m_graph, node, JSCell_structureID))
240 DFG_CRASH(m_graph, node, toCString("AI-clobberize disagreement; AI says ", m_state.clobberState(), " while clobberize says ", writeSet(m_graph, node)).data());
241 }
242 if (m_verbose) {
243 dataLogF(" tail regs: ");
244 m_interpreter.dump(WTF::dataFile());
245 dataLogF("\n");
246 }
247 m_changed |= m_state.endBasicBlock();
248
249 if (m_verbose) {
250 dataLog(" tail vars: ", block->valuesAtTail, "\n");
251 if (m_graph.m_form == SSA)
252 dataLog(" head regs: ", nodeValuePairListDump(block->ssa->valuesAtTail), "\n");
253 }
254 }
255
256 void performForwardCFA()
257 {
258 ++m_count;
259 if (m_verbose)
260 dataLogF("CFA [%u]\n", m_count);
261
262 for (BlockIndex blockIndex = 0; blockIndex < m_graph.numBlocks(); ++blockIndex)
263 performBlockCFA(m_graph.block(blockIndex));
264 }
265
266private:
267 InPlaceAbstractState m_state;
268 AbstractInterpreter<InPlaceAbstractState> m_interpreter;
269 BlockSet m_blocksWithOSR;
270
271 const bool m_verbose;
272
273 bool m_changed;
274 unsigned m_count;
275};
276
277bool performCFA(Graph& graph)
278{
279 return runPhase<CFAPhase>(graph);
280}
281
282} } // namespace JSC::DFG
283
284#endif // ENABLE(DFG_JIT)
Note: See TracBrowser for help on using the repository browser.