← Back to System Status

Orchestrator Tasks

Dashboard / System Status / Documentation / Orchestrator Tasks

Orchestrator Tasks Reference

BFS Marketing Automation - Complete Task Documentation

Last Updated: November 25, 2025

Overview

The orchestrator manages 11 automated tasks that run on schedules via cron jobs. Each task is a standalone Python script that can be run manually or automatically.

Total Tasks: 11 (added 2 video workflow tasks Nov 24) Execution Tracking: All runs logged in orchestrator_runs database table Manual Triggers: Available via dashboard automation banners

Task List

1. Content Generation

File: orchestrator/tasks/generate_content.py Schedule: Monday 6:00 AM Duration: ~2-3 minutes Status: ✅ Active

Purpose: Generates ~65 social media posts for the upcoming week using Claude AI.

What it does: 1. Queries database for week start date 2. Checks if content already generated for this week 3. Calls ContentGenerator to create posts 4. Generates mix of 80% image posts + 20% video posts 5. Creates platform-specific captions for each post 6. Assigns posts to offices and social accounts 7. Sets all posts to pending_approval status 8. Records execution in orchestrator_runs table

Manual Execution:

python3 orchestrator/tasks/generate_content.py --force

Configuration (orchestrator/config.py): - CONTENT_GENERATION_DAY = 1 (Monday) - CONTENT_GENERATION_HOUR = 6 (6 AM) - POSTS_PER_WEEK_TARGET = 65

Output: - 53 office posts (26 offices × 2-3 each) - 12 social media posts (Instagram, TikTok, YouTube, Twitter) - All posts saved to content_library table - Office assignments saved to office_assignments table

Notifications: - Teams notification sent when complete (when Teams enabled) - Dashboard shows posts in "Pending Approval" section

2. Image Generation (NEW - Nov 20)

File: orchestrator/tasks/generate_pending_images.py Schedule: 3x daily (6:00 AM, 12:00 PM, 6:00 PM) Duration: ~3-5 minutes (depends on post count) Status: ✅ Active

Purpose: Automatically generates platform-specific images for approved posts using DALL-E 3.

What it does: 1. Queries database for posts in pending_design status 2. For each post, reads platform_images metadata (sizes selected in Stage 1) 3. Generates requested images via DALL-E 3: - Square (1024×1024) for Facebook, Instagram, Twitter - Portrait (1024×1792) for Stories, TikTok, Shorts 4. Saves images to data/images/ directory 5. Updates platform_images column with image paths 6. Changes post status to design_complete 7. Records execution and image count in orchestrator_runs table

Manual Execution:

python3 orchestrator/tasks/generate_pending_images.py --force

Manual Trigger: Green "Generate Now" button on Pending Design page

Configuration: - No specific config (runs based on pending posts) - Cost tracked per image: $0.04 per size

Output: - Images saved as: data/images/POSTID_square.png, POSTID_portrait.png - Database updated with image paths - Posts moved to design_complete status

Cost: - Square: $0.04 - Portrait: $0.04 - Both: $0.08 per post

3. Metricool Scheduling (NEW - Nov 20)

File: orchestrator/tasks/publish_to_metricool.py Schedule: Every hour Duration: ~1-2 minutes Status: 🧪 Test Mode

Purpose: Automatically schedules approved posts to Metricool for multi-platform publishing.

What it does (when enabled): 1. Queries database for posts in approved status 2. For each post, prepares scheduling data 3. Calls Metricool API to schedule post 4. Sets optimal publish time based on platform 5. Updates post status to published 6. Records scheduling details in database

What it does (test mode - current): 1. Queries database for posts in approved status 2. Prepares scheduling data (no API call) 3. Logs what WOULD be scheduled 4. Does NOT change post status 5. Records test execution in orchestrator_runs table

Manual Execution:

python3 orchestrator/tasks/publish_to_metricool.py --force

Manual Trigger: Orange "Schedule Now" button on Approved page

Configuration (orchestrator/config.py): - METRICOOL_AUTO_PUBLISH = False (test mode) - METRICOOL_PUBLISH_APPROVED_POSTS = True - METRICOOL_PUBLISH_INTERVAL_HOURS = 1 - METRICOOL_USER_ID = None (set when enabling)

Enabling Auto-Publish: 1. Set METRICOOL_AUTO_PUBLISH = True in config 2. Add METRICOOL_USER_ID and METRICOOL_API_KEY to credentials 3. No restart needed (takes effect on next run)

Output (when enabled): - Posts scheduled in Metricool - Post status updated to published - Scheduling metadata recorded

4. Approval Monitoring

File: orchestrator/tasks/check_approvals.py Schedule: Every hour at :15 (e.g., 1:15, 2:15, 3:15) Duration: <1 minute Status: ✅ Active

Purpose: Monitors pending approvals and sends reminders if posts are waiting too long.

What it does: 1. Queries database for posts in pending_approval status 2. Checks how long each post has been waiting 3. If waiting > 24 hours: Send reminder notification 4. If waiting > 48 hours: Send escalation notification 5. Records monitoring status in orchestrator_runs table

Manual Execution:

python3 orchestrator/tasks/check_approvals.py

Configuration (orchestrator/config.py): - APPROVAL_REMINDER_THRESHOLD = 24 (hours) - APPROVAL_ESCALATION_THRESHOLD = 48 (hours)

Notifications: - 24h reminder: "Posts still awaiting approval" - 48h escalation: "URGENT: Approval workflow delayed"

5. Designer Notifications

File: orchestrator/tasks/notify_designers.py Schedule: 3x daily (6:00 AM, 12:00 PM, 6:00 PM) Duration: <1 minute Status: ✅ Active (less relevant with automated image generation)

Purpose: Notifies designers when posts need manual design work (for posts with "background only" option).

What it does: 1. Queries database for posts in pending_design status 2. Filters for posts marked "no_text" or custom design needed 3. Sends Teams notification with count of posts needing design 4. Records notification status in orchestrator_runs table

Manual Execution:

python3 orchestrator/tasks/notify_designers.py --force

Configuration: - DESIGN_REMINDER_THRESHOLD = 24 (hours) - DESIGN_ESCALATION_THRESHOLD = 48 (hours)

Note: With automated DALL-E generation, this is primarily for posts needing manual Canva designs.

6. Design Completion Checks

File: orchestrator/tasks/check_designs.py Schedule: 3x daily (6:45 AM, 12:45 PM, 6:45 PM) Duration: <1 minute Status: ✅ Active

Purpose: Checks when designs are completed and notifies for Stage 2 review.

What it does: 1. Queries database for posts in design_complete status 2. Sends Teams notification when designs are ready for review 3. Records check status in orchestrator_runs table

Manual Execution:

python3 orchestrator/tasks/check_designs.py --force

Notifications: - "Designs complete - ready for Stage 2 approval" - Includes count of posts ready for review

7. Create SharePoint Video Folders (NEW - Nov 24)

File: orchestrator/tasks/create_video_folders.py Schedule: Every 5 minutes Duration: <1 minute Status: ✅ Active

Purpose: Automatically creates SharePoint folders for video posts that are approved and awaiting video upload.

What it does: 1. Queries database for posts in pending_video_upload status 2. For each post, checks if SharePoint folder already exists 3. Creates folder structure: /Video Content/Post-{id}-{name}/ 4. Uses video concept to generate friendly folder names 5. Handles nested folder creation (ensures parent folder exists) 6. Records folder creation status in orchestrator_runs table

Manual Execution:

python3 orchestrator/tasks/create_video_folders.py --verbose

Configuration: - No specific config (runs based on pending posts) - Folder naming: Post-{post_id}-{concept-name}

Output: - SharePoint folders created within 5 minutes of approval - Staff can immediately upload videos via WiFi - Folder path logged for tracking

Why 5 Minutes: - Ensures folders ready quickly after approval - Allows staff to upload videos immediately - More frequent than video sync (15 min) to avoid race conditions

8. Sync SharePoint Videos (NEW - Nov 24)

File: orchestrator/tasks/sync_sharepoint_videos.py Schedule: Every 15 minutes Duration: ~1-2 minutes Status: ✅ Active

Purpose: Automatically detects and syncs videos uploaded directly to SharePoint by staff.

What it does: 1. Queries database for posts in pending_video_upload status 2. For each post, scans SharePoint folder for videos 3. Parses filenames to extract office and platform info: - Format: {Office}_{Platform}.mp4 - Example: Erie_General.mp4, Indianapolis_Reels.mp4 4. Skips already-synced videos (checks existing records) 5. For new videos: - Creates video_files record in database - Extracts video metadata (duration, resolution, codec, bitrate) - Generates thumbnail (400px width) - Updates post video_count 6. Moves post to video_review status when videos found 7. Records sync execution in orchestrator_runs table

Manual Execution:

python3 orchestrator/tasks/sync_sharepoint_videos.py --verbose

Configuration: - No specific config (runs based on pending posts) - Filename parsing: {Office}_{Platform}.ext

Output: - Videos synced from SharePoint to database - Thumbnails generated automatically - Posts moved to review queue - Staff notified when videos ready for review

Graceful Handling: - Continues if folder doesn't exist yet - Skips unparseable filenames (logs warning) - Handles metadata extraction failures - Works even if SharePoint unavailable

Why 15 Minutes: - Balance between responsiveness and system load - Gives staff time to upload multiple videos - Aligns with typical upload workflow timing

9. SharePoint Backup

File: orchestrator/tasks/backup_approved_posts.py Schedule: Daily 11:00 PM Duration: ~2-5 minutes (depends on post count) Status: ⏳ Pending Microsoft propagation

Purpose: Backs up all approved posts to SharePoint for compliance and disaster recovery.

What it does: 1. Queries database for newly approved posts (since last backup) 2. For each post, prepares backup package: - Post metadata (JSON) - All platform images - Video files (if applicable) 3. Uploads to SharePoint organized by year/week 4. Records backup status in database 5. Logs backup details in orchestrator_runs table

Manual Execution:

python3 orchestrator/tasks/backup_approved_posts.py --verbose

Configuration: - SharePoint site: Configured in credentials.env - Folder structure: Shared Documents/BFS Marketing/{YEAR}/Week-{WEEK}/

Output: - Files backed up to SharePoint - Backup metadata recorded in database - Daily backup summary in logs

Status: Waiting for SharePoint permissions to propagate (expected Nov 22).

10. Weekly Analytics

File: orchestrator/tasks/weekly_analytics.py Schedule: Friday 5:00 PM Duration: ~1 minute Status: ✅ Active

Purpose: Generates weekly performance summary and sends to marketing team.

What it does: 1. Queries database for current week's posts 2. Calculates metrics: - Posts generated vs. target - Approval rates (Stage 1 and Stage 2) - Rejection reasons (top 5) - Workflow timing (average time for each stage) - Bottlenecks detected - System health summary 3. Sends Teams notification with summary 4. Records analytics in analytics_history table 5. Logs execution in orchestrator_runs table

Manual Execution:

python3 orchestrator/tasks/weekly_analytics.py --force

Configuration (orchestrator/config.py): - ANALYTICS_DAY = 5 (Friday) - ANALYTICS_HOUR = 17 (5 PM)

Metrics Tracked: - Total posts generated - Stage 1 approval rate - Stage 2 approval rate - Average review time - Top rejection reasons - Workflow bottlenecks - System health indicators

11. Health Check

File: orchestrator/tasks/health_check.py Schedule: Every 30 minutes Duration: <1 minute Status: ✅ Active

Purpose: Monitors system health and alerts on issues.

What it does: 1. Checks database size and integrity 2. Checks disk usage 3. Checks memory usage 4. Checks for stuck workflows (posts waiting > 4 hours) 5. Checks recent system errors in logs 6. Sends alert if issues detected 7. Records health status in orchestrator_runs table

Manual Execution:

python3 orchestrator/tasks/health_check.py --verbose

Configuration (orchestrator/config.py): - HEALTH_CHECK_INTERVAL = 30 (minutes) - MAX_DB_SIZE_MB = 500 - MAX_DISK_USAGE_PERCENT = 90 - MAX_MEMORY_USAGE_PERCENT = 90 - STUCK_WORKFLOW_THRESHOLD = 4 (hours)

Alerts Sent For: - Database > 500MB - Disk usage > 90% - Memory usage > 90% - Posts stuck in workflow > 4 hours - Recent system errors

Task Execution

Cron Installation

Install all tasks:

cd ~/bfs-projects/Marketing
./orchestrator/setup_cron.sh

View installed cron jobs:

crontab -l | grep "BFS Marketing"

Manual Execution

All tasks support manual execution with optional flags:

# Force run (bypass schedule checks)
python3 orchestrator/tasks/TASK_NAME.py --force

# Verbose output (detailed logging)
python3 orchestrator/tasks/TASK_NAME.py --verbose

# Both
python3 orchestrator/tasks/TASK_NAME.py --force --verbose

Execution Tracking

View recent executions:

python3 -c "
from shared.database import get_db
db = get_db()
runs = db.get_recent_orchestrator_runs(limit=20)
for run in runs:
    print(f'{run[\"task_name\"]}: {run[\"status\"]} ({run[\"started_at\"]})')
"

Logs

All task output is logged to:

logs/orchestrator/cron.log

View logs:

tail -f ~/bfs-projects/Marketing/logs/orchestrator/cron.log

Task Dependencies

Workflow Sequence

Photo Workflow:

generate_content.py (Task 1)
    ↓ Creates photo posts with pending_approval status
[User reviews text + selects platforms in Stage 1]
    ↓ Approves with platform selection
generate_pending_images.py (Task 2)
    ↓ Generates images, sets design_complete status
[User reviews complete posts in Stage 2]
    ↓ Approves posts
publish_to_metricool.py (Task 3)
    ↓ Schedules to Metricool, sets published status
backup_approved_posts.py (Task 9)
    ↓ Backs up to SharePoint

Video Workflow (Updated Nov 25 - Release Management):

generate_content.py (Task 1)
    ↓ Creates video posts with pending_approval status
[User reviews video concept/script in Stage 1]
    ↓ Approves video concept → status: pending_video_upload
create_video_folders.py (Task 7)
    ↓ Creates SharePoint folder within 5 minutes
[Staff uploads videos to SharePoint via WiFi or dashboard]
sync_sharepoint_videos.py (Task 8)
    ↓ Detects uploads, videos appear in Video Sorter
[User assigns videos in Video Sorter: Post → Release → Office → Platform]
    ↓ Videos assigned → post status: video_library (permanent)
[User manages Releases in Video Post Detail page]
    ↓ Create/Edit/Delete Releases as needed
    ↓ Update individual video status (Available/Scheduled/Published/Archived)
publish_to_metricool.py (Task 3)
    ↓ Schedules to Metricool, update video status to Published
backup_approved_posts.py (Task 9)
    ↓ Backs up to SharePoint (including videos)

Note: With Release Management (Nov 25), posts stay in video_library status permanently. Individual videos track their own status. Same concept can be scheduled multiple times via different Releases.

Independent Tasks

These run independently across both workflows: - check_approvals.py (Task 4) - Monitors any time - notify_designers.py (Task 5) - Notifies as needed - check_designs.py (Task 6) - Checks completion - weekly_analytics.py (Task 10) - Weekly summary - health_check.py (Task 11) - Continuous monitoring

Troubleshooting

Task Not Running

  1. Check cron installation: bash crontab -l | grep "TASK_NAME"

  2. Check cron service: bash sudo systemctl status cron

  3. Check logs: bash grep "TASK_NAME" logs/orchestrator/cron.log

Task Failing

  1. Run manually with verbose: bash python3 orchestrator/tasks/TASK_NAME.py --verbose

  2. Check database: bash python3 -c "from shared.database import get_db; db = get_db(); print('✓ Database OK')"

  3. Check recent runs: bash python3 -c "from shared.database import get_db; db = get_db(); runs = db.get_recent_orchestrator_runs(); print([r for r in runs if r['task_name'] == 'TASK_NAME'][:3])"

Disabling a Task

Comment out in crontab:

crontab -e
# Add # before the task line to disable

Future Enhancements

Planned additions: - A/B testing task - Performance optimization task - Social media monitoring task - Automated response generation task - Multi-language content generation

Reference: This document consolidates information from the codebase and serves as the definitive guide to orchestrator tasks.

BFS Marketing Automation Documentation

For technical support, contact your system administrator