Transcoding failed: Missing lock for job

[kontrollanten]

Describe the current behavior

{
    "err": "Error: EIO: i/o error, read",
    "level": "error",
    "message": "Error in controller.",
    "timestamp": "2024-05-08T18:12:09.366Z"
}
{
    "level": "error",
    "message": "Error in job worker video-transcoding.",
    "err": {
        "stack": "Error: Missing lock for job de73a010-53c6-4142-91fa-0b17894c0f59. failed\n    at Scripts.finishedErrors (/var/www/peertube/versions/peertube-v6.1.0-c1/node_modules/bullmq/dist/cjs/classes/scripts.js:275:24)\n    at Job.moveToFailed (/var/www/peertube/versions/peertube-v6.1.0-c1/node_modules/bullmq/dist/cjs/classes/job.js:439:32)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async handleFailed (/var/www/peertube/versions/peertube-v6.1.0-c1/node_modules/bullmq/dist/cjs/classes/worker.js:380:21)\n    at async Worker.retryIfFailed (/var/www/peertube/versions/peertube-v6.1.0-c1/node_modules/bullmq/dist/cjs/classes/worker.js:584:24)",
        "message": "Missing lock for job de73a010-53c6-4142-91fa-0b17894c0f59. failed"
    },
    "timestamp": "2024-05-08T18:12:45.763Z"
}
{
    "level": "error",
    "message": "Cannot execute job de73a010-53c6-4142-91fa-0b17894c0f59 in queue video-transcoding.",
    "payload": {
        "type": "new-resolution-to-hls",
        "videoUUID": "02f4fe9f-763e-43bb-b087-ae7142253ed8",
        "resolution": 1080,
        "fps": 25,
        "copyCodecs": true,
        "isNewVideo": true,
        "deleteWebVideoFiles": true
    },
    "err": {
        "stack": "TypeError: Cannot read properties of undefined (reading 'getVideoOrStreamingPlaylist')\n    at handleHLSJob (file:///var/www/peertube/versions/peertube-v6.1.0-c1/dist/core/lib/job-queue/handlers/video-transcoding.js:71:57)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async Object.processVideoTranscoding [as video-transcoding] (file:///var/www/peertube/versions/peertube-v6.1.0-c1/dist/core/lib/job-queue/handlers/video-transcoding.js:34:9)\n    at async Object.wrapPromiseFun (file:///var/www/peertube/versions/peertube-v6.1.0-c1/dist/core/lib/plugins/hooks.js:8:24)\n    at async Worker.processJob (/var/www/peertube/versions/peertube-v6.1.0-c1/node_modules/bullmq/dist/cjs/classes/worker.js:395:28)\n    at async Worker.retryIfFailed (/var/www/peertube/versions/peertube-v6.1.0-c1/node_modules/bullmq/dist/cjs/classes/worker.js:584:24)",
        "message": "Cannot read properties of undefined (reading 'getVideoOrStreamingPlaylist')"
    },
    "timestamp": "2024-05-08T18:12:45.839Z"
}

I don't think the EIO error is related since we see a lot of them in the logs, but since it was close in time it may be related.

Steps to reproduce

Describe the expected behavior

No response

Additional information

• PeerTube instance:

  • URL:
  • Version: 6.1.0
  • NodeJS version: 18.20.2
  • Ffmpeg version: 4.4.2
  • Redis version: 7.2.3

• Browser name, version and platforms on which you could reproduce the bug:

• Link to browser console log if relevant:

• Link to server log if relevant (journalctl or /var/www/peertube/storage/logs/):

[Chocobozzz]

Hi,

I think the Cannot read properties of undefined error is related to the broken job Missing lock for job.

Did you have an issue with Redis? (EIO: i/o error, read is also a clue)

[kontrollanten]

Not any known issues with redis. When I check the redis logs there's no errors and the container has been up for months.

[Chocobozzz]

Do you have many Missing lock for job errors in your logs?

[kontrollanten]

No, we had 10 occurrences during a few days. During the same days we had issues with connection to our S3 provider, but that shouldn't affect transcoding jobs?