Skip to content

getWorldHandlers() called at module load time breaks OpenNext/Lambda deployments with custom World #825

@gmathieu

Description

@gmathieu

Environment

  • SST v3.17.23 with sst.aws.Nextjs component
  • OpenNext v3.6.6
  • @workflow/world-postgres@4.1.0-beta.28
  • workflow@4.0.1-beta.49

Problem

When deploying to AWS Lambda via OpenNext, we get:

Error: Cannot find module '@workflow/world-postgres'

Unrelated issue reported with the same error message: #689.

Root Cause

getWorldHandlers() is called at module load time when importing workflow/runtime:

const stepHandler = getWorldHandlers().createQueueHandler(
'__wkf_step_',
async (message_, metadata) => {

This runs before instrumentation.ts's register() function has a chance to call setWorld().

When getWorldHandlers() finds an empty cache, it calls createWorld():

const mod = require(targetWorld);

This dynamic require() with a variable path (WORKFLOW_TARGET_WORLD=@workflow/world-postgres) cannot be traced by Next.js/OpenNext bundler, so the module isn't included in the Lambda bundle.

Our Setup

src/instrumentation.ts:

// open-next does not support dynamic imports: https://opennext.js.org/aws/common_issues#cannot-find-module-chunksxxxxjs-error
import { createWorld } from '@workflow/world-postgres'
import { setWorld } from 'workflow/runtime'

export async function register() {
  const world = createWorld()
  setWorld(world)

  // This only runs in local development. In AWS, we start the queue listener in the workflow-worker service
  if (process.env.IS_LOCAL_STAGE) await world.start()
}

infra/environment.ts:

WORKFLOW_TARGET_WORLD: '@workflow/world-postgres',

pkgs/workflow-worker/src/worker.ts (if you're curious):

import { createWorld } from '@workflow/world-postgres'
import * as z from 'zod'

const schema = z.object({
  WORKFLOW_LOCAL_BASE_URL: z.string(),
  WORKFLOW_POSTGRES_JOB_PREFIX: z.string(),
  WORKFLOW_POSTGRES_URL: z.string(),
  WORKFLOW_POSTGRES_WORKER_CONCURRENCY: z.string(),
  WORKFLOW_TARGET_WORLD: z.string(),
})

async function main() {
  console.log('[Worker] Starting workflow worker process...')

  const result = schema.safeParse(process.env)
  if (result.error) {
    console.error(
      '[Worker] Failed to parse environment variables:',
      result.error.format(),
    )
    process.exit(1)
  }

  const world = createWorld()
  await world.start()
}

void main()

The static import of @workflow/world-postgres should make it available, but the module initializes before register() runs.

Workaround

Downgrade to @workflow/world-postgres@4.1.0-beta.16 and workflow@4.0.1-beta.27. We don't know exactly when the behavior changed (maybe after #544). We know these versions are working in a different deployment.

Attempted Workarounds

We tried forcing the bundle to include the package via next.config.ts:

outputFileTracingIncludes: {
  '/instrumentation': ['./node_modules/@workflow/**/*'],
},

This requires node-linker=hoisted in .npmrc (pnpm symlinks break OpenNext's file copying). However, including all @workflow packages exceeds Lambda's 250MB unzipped size limit. Narrowing to just @workflow/world-postgres doesn't help since the dynamic require() pulls in transitive dependencies that aren't bundled.

Suggested Fix

Lazy-evaluate getWorldHandlers() in step-handler.ts instead of calling it at module load time, or provide a way to ensure the world is set before any workflow modules initialize.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions