Running now
| Source | Phase | Now | Companies | Found | New | Jobs a minute | Time vs usual | Quiet for | Last 10 runs |
|---|
Finished this cycle
Scraper box
- In use now
- –
- Peak
- –
- Swap used
- –
- Free on the box
- –
- Kernel-counted memory
- –
- Throttled by the memory cap
- –
- Out-of-memory kills
- –
- Load
- –
- Disk free
- –
- Goroutines
- –
- Log file
- –
Memory in use is the daemon's own; kernel-counted memory adds the page cache charged to it, and that is the figure the throttle line and the kill line act on. The dashed lines are the Go memory limit, the throttle line and the point where the daemon is killed.
Files
| File | Status | Rows: new, updated, expired, no-op, failed | India | Took | Waited | Rows/min | Cost | Attempts |
|---|
Click a file for its attempts, the row range each one covered, the reason it stopped, and the worker's log lines for that file.
Where are tonight's jobs? scraper files, from the box to the index
Queue
When the blue line sits above the orange one, ingest is falling behind.
Flat and low is healthy. A steady climb means nothing is being picked up.
Ingest worker log
Is expiry really running? per source, newest expiry pass
Every enabled source with expiry on is listed. A source that failed before its expiry pass, or did not run in the latest cycle, shows its newest older pass and says so. Closed is what the pass sent to the index; Landed is what the index confirmed when the pass finished. Seen counts rows the pass recorded as sighted, not the size of the walk.
| Source | Active jobs | Seen | Expiry | Closed | Landed | Held back | What it means |
|---|
Scraper jobs added and expired, last 14 days
Scraper sources only; the linkedin_vendor feed is left out. Added counts documents created that day, not re-indexed ones. Expired is bucketed by the day the closure was proven, so a closure delivered late lands on an earlier day.
View as table
| Day | Added | Expired |
|---|
Corpus
- Active jobs
- –
- Active, India
- –
- Expired
- –
- All jobs
- –
- Companies
- –
- Active, scrapers only
- –
- Expired, scrapers only
- –
Totals are context, not a health signal.