Jobs Collections

Overview

Two collections manage background operations: media_jobs for BullMQ job records, and hitl_sessions for human-in-the-loop login requests.

media_jobs

Tracks BullMQ background jobs for media processing operations. Each job record corresponds to a queue entry in Redis.

Purpose

Track status of background media operations
Store job metadata, progress, and results
Enable job retry and error handling
Link jobs to creator profiles and media items
Provide audit trail for all media operations

Job Types

Type	Description	Worker Queue
`scrape_profile`	Scrape platform for new content and metrics	`scrape-jobs`
`publish_post`	Publish scheduled post to platform(s)	`media-jobs`
`apply_watermark`	Add watermark to image/video using ImageMagick	`media-jobs`
`create_teaser`	Generate video preview using FFmpeg	`media-jobs`
`crop_media`	Crop/resize media file	`media-jobs`
`compress_media`	Compress media for faster delivery	`media-jobs`

Key Fields

Field	Type	Description
`id`	UUID	Primary key
`job_type`	String	One of the job types listed above
`status`	String	`queued`, `processing`, `completed`, `failed`
`creator_profile_id`	Foreign Key	Optional link to `creator_profiles`
`media_id`	Foreign Key	Optional link to `scraped_media`
`scheduled_post_id`	Foreign Key	Optional link to `scheduled_posts`
`bull_job_id`	String	BullMQ Redis job ID for status tracking
`params`	JSON	Job-specific parameters
`progress`	Integer	0-100 completion percentage
`result`	JSON	Job output data on completion
`error_message`	Text	Error details if status=failed
`retry_count`	Integer	Number of retry attempts
`started_at`	DateTime	When job processing began
`completed_at`	DateTime	When job finished (success or failure)
`created_at`	DateTime	Job creation timestamp

Example Queries

Create Scrape Job

await use_mcp_tool({
  server_name: "directus",
  tool_name: "create-item",
  arguments: {
    collection: "media_jobs",
    data: {
      job_type: "scrape_profile",
      creator_profile_id: "profile-uuid",
      status: "queued",
      params: {
        platform: "onlyfans",
        scrape_type: "full",
        max_items: 100
      }
    }
  }
});

curl -X POST "https://geniehelper.com/api/directus/items/media_jobs" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "job_type": "scrape_profile",
    "creator_profile_id": "profile-uuid",
    "status": "queued",
    "params": {
      "platform": "onlyfans",
      "scrape_type": "full",
      "max_items": 100
    }
  }'

Monitor Job Status

await use_mcp_tool({
  server_name: "directus",
  tool_name: "read-item",
  arguments: {
    collection: "media_jobs",
    id: "job-uuid",
    fields: ["id", "job_type", "status", "progress", "error_message"]
  }
});

curl -X GET "https://geniehelper.com/api/directus/items/media_jobs/JOB_ID" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -G \
  --data-urlencode 'fields=id,job_type,status,progress,error_message'

List Recent Jobs

await use_mcp_tool({
  server_name: "directus",
  tool_name: "read-items",
  arguments: {
    collection: "media_jobs",
    fields: ["id", "job_type", "status", "progress", "created_at"],
    sort: ["-created_at"],
    limit: 20
  }
});

curl -X GET "https://geniehelper.com/api/directus/items/media_jobs" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -G \
  --data-urlencode 'sort=-created_at' \
  --data-urlencode 'limit=20'

Find Failed Jobs

await use_mcp_tool({
  server_name: "directus",
  tool_name: "read-items",
  arguments: {
    collection: "media_jobs",
    fields: ["id", "job_type", "error_message", "retry_count", "created_at"],
    filter: {
      status: { _eq: "failed" },
      created_at: { _gte: "$NOW(-24 hours)" }
    },
    sort: ["-created_at"]
  }
});

curl -X GET "https://geniehelper.com/api/directus/items/media_jobs" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -G \
  --data-urlencode 'filter[status][_eq]=failed' \
  --data-urlencode 'filter[created_at][_gte]=$NOW(-24 hours)' \
  --data-urlencode 'sort=-created_at'

hitl_sessions

Human-in-the-loop login requests displayed as yellow dashboard alerts when automated scraping requires manual intervention.

Purpose

Request manual login when platform cookies expire
Display browser extension flow prompts in dashboard
Track session resolution status
Coordinate between automated scraping and human intervention

Key Fields

Field	Type	Description
`id`	UUID	Primary key
`creator_profile_id`	Foreign Key	Links to `creator_profiles`
`platform`	String	Platform requiring login
`session_type`	String	`login_required`, `cookies_expired`, `2fa_required`
`status`	String	`pending`, `in_progress`, `resolved`, `cancelled`
`alert_message`	Text	Message shown in dashboard alert
`resolution_method`	String	How resolved: `browser_extension`, `manual_cookies`, `oauth`
`resolved_at`	DateTime	When session was resolved
`expires_at`	DateTime	Auto-cancel if not resolved by this time
`created_at`	DateTime	Session creation timestamp

Dashboard Integration

HITL sessions trigger yellow banner alerts in the dashboard:

// Dashboard polling logic
const activeSessions = await directus.items('hitl_sessions').readByQuery({
  filter: {
    creator_profile_id: currentUser.id,
    status: { _in: ['pending', 'in_progress'] }
  }
});

if (activeSessions.data.length > 0) {
  showYellowBanner(activeSessions.data[0].alert_message);
}

Example Queries

Create HITL Session

await use_mcp_tool({
  server_name: "directus",
  tool_name: "create-item",
  arguments: {
    collection: "hitl_sessions",
    data: {
      creator_profile_id: "profile-uuid",
      platform: "onlyfans",
      session_type: "cookies_expired",
      status: "pending",
      alert_message: "OnlyFans login required. Click to connect using browser extension.",
      expires_at: "$NOW(+24 hours)"
    }
  }
});

curl -X POST "https://geniehelper.com/api/directus/items/hitl_sessions" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "creator_profile_id": "profile-uuid",
    "platform": "onlyfans",
    "session_type": "cookies_expired",
    "status": "pending",
    "alert_message": "OnlyFans login required. Click to connect using browser extension."
  }'

List Active Sessions

await use_mcp_tool({
  server_name: "directus",
  tool_name: "read-items",
  arguments: {
    collection: "hitl_sessions",
    fields: ["id", "platform", "session_type", "alert_message", "status"],
    filter: {
      status: { _in: ["pending", "in_progress"] },
      expires_at: { _gte: "$NOW" }
    }
  }
});

curl -X GET "https://geniehelper.com/api/directus/items/hitl_sessions" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -G \
  --data-urlencode 'filter[status][_in]=pending,in_progress' \
  --data-urlencode 'filter[expires_at][_gte]=$NOW'

Resolve Session

await use_mcp_tool({
  server_name: "directus",
  tool_name: "update-item",
  arguments: {
    collection: "hitl_sessions",
    id: "session-uuid",
    data: {
      status: "resolved",
      resolution_method: "browser_extension",
      resolved_at: new Date().toISOString()
    }
  }
});

curl -X PATCH "https://geniehelper.com/api/directus/items/hitl_sessions/SESSION_ID" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "status": "resolved",
    "resolution_method": "browser_extension",
    "resolved_at": "2026-03-04T12:00:00Z"
  }'

creator_profiles - Platform accounts that jobs operate on
scraped_media - Content processed by media jobs
scheduled_posts - Posts published by publish_post jobs
platform_sessions - Browser cookies that resolve HITL sessions

Workflow Integration

Scraping with HITL Flow

User clicks “Let’s Go” scrape button on dashboard
System checks platform_sessions for valid cookies
If cookies exist: Create media_jobs entry with type scrape_profile
If no cookies: Create hitl_sessions entry instead
Dashboard shows yellow banner: “Login required. Click to connect.”
User completes login via browser extension
Extension captures cookies → stored in platform_sessions
HITL session marked resolved
System creates media_jobs entry, scraping proceeds

Media Processing Flow

User clicks crop/watermark/teaser button in Media Library
Frontend creates media_jobs entry via POST /api/queue/enqueue
BullMQ worker picks up job from Redis queue
Worker updates job status and progress fields
On completion: job result contains output file URLs
Frontend polls job status and displays result

Best Practices

Poll job status - Use bull_job_id to track BullMQ queue position
Implement retry logic - Jobs with retry_count < 3 can be retried
Clean up completed jobs - Archive jobs older than 30 days
Monitor HITL expiry - Auto-cancel sessions after 24 hours
Handle concurrent HITLs - Only show one HITL alert per platform at a time

BullMQ Integration

The media worker (media-worker/index.js) processes jobs from Redis queues:

// Queue configuration
const scrapeQueue = new Queue('scrape-jobs', { connection: redis });
const mediaQueue = new Queue('media-jobs', { connection: redis });

// Worker processors
const scrapeWorker = new Worker('scrape-jobs', async (job) => {
  // Update media_jobs status
  await directus.items('media_jobs').updateOne(job.data.recordId, {
    status: 'processing',
    bull_job_id: job.id
  });
  
  // Process job...
});

Endpoints

MCP Tools

Collections

Overview

media_jobs

Purpose

Job Types

Key Fields

Example Queries

Create Scrape Job

Monitor Job Status

List Recent Jobs

Find Failed Jobs

hitl_sessions

Purpose

Key Fields

Dashboard Integration

Example Queries

Create HITL Session

List Active Sessions

Resolve Session

Workflow Integration

Scraping with HITL Flow

Media Processing Flow

Best Practices

BullMQ Integration

See Also

​Overview

​media_jobs

​Purpose

​Job Types

​Key Fields

​Example Queries

​Create Scrape Job

​Monitor Job Status

​List Recent Jobs

​Find Failed Jobs

​hitl_sessions

​Purpose

​Key Fields

​Dashboard Integration

​Example Queries

​Create HITL Session

​List Active Sessions

​Resolve Session

​Related Collections

​Workflow Integration

​Scraping with HITL Flow

​Media Processing Flow

​Best Practices

​BullMQ Integration

​See Also

Overview

media_jobs

Purpose

Job Types

Key Fields

Example Queries

Create Scrape Job

Monitor Job Status

List Recent Jobs

Find Failed Jobs

hitl_sessions

Purpose

Key Fields

Dashboard Integration

Example Queries

Create HITL Session

List Active Sessions

Resolve Session

Related Collections

Workflow Integration

Scraping with HITL Flow

Media Processing Flow

Best Practices

BullMQ Integration

See Also