Manual criteria — fields map to JobSearchCriteriaV2JobSearchCriteriaV3 and Search runs the exact DB_LI search for the selected pipeline. Seniority is not a field — put it in role_keywordsdirect_roles (e.g. "senior software engineer"). Comma-separate multi-value fields.
How each field is searched + the scoring weights (v2)

Every criteria field maps to one or more document fields in the jobfetcher_jobs index, and contributes either to the hard filter gate (which jobs match at all), the scoring tier (which jobs rank higher), or both. Scoring is an additive budget that sums to ≈ 1.0; ratio buckets divide their weight by the term count, so a long list never out-ranks a short one.

Field → document-field mapping

Criteria field Role in the query Document fields hit
primary_role Scoring only (Tier 1) title (match_phrase, analyzed)
role_keywords Hard gate + scoring (Tier 2 variant) Gate: title.keyword (word-bounded substring — no analyzer) · Scoring: title (match_phrase)
skill_keywords Hard gate + ratio scoring title, description.full, description.requirements, description.qualifications
qualifier_keywords Hard gate (+ optional ratio scoring via the toggle) title, description.full, description.about_company
disambiguating_context Optional gate + optional ratio scoring (both off by default) description.full
anti_keywords Hard must_not title only (match_phrase) — narrow on purpose, so a JD that just mentions an anti term in passing isn't excluded
company_names Hard filter company_name (match_phrase)
locations Hard filter location.city / location.state / location.country — AND'd within an entry, OR'd across entries
workplace_type / is_remote Hard filter OR of the document's is_remote and workplace_type fields (different scrapers populate different fields)
min/max_experience_years Hard filter min_experience_years range — only jobs that declare an experience floor are filtered; jobs without one pass through
min_salary Soft scoring bonus (NOT a gate) compensation fields
posted_within_days Hard filter date_posted range
job_type Hard filter job_type (term)
always-on Hard gate is_expired: false
always-on Soft scoring date_posted gauss decay — fresher jobs rank higher (offset 7d, scale 30d)

Scoring weights (sum ≈ 1.0)

Bucket Weight When it fires Formula
Skills (ratio) W_SKILLS = 0.60 / 0.45 Always, when any skill_keywords is supplied. 0.60 when context scoring is OFF; 0.45 when ON (the remaining 0.15 goes to context) W_SKILLS × (skills_matched / skills_total)
Disambiguating context (ratio) W_GOOD = 0.15 Only when "Score disambiguating_context" is ON W_GOOD × (context_matched / context_total)
Qualifier (ratio) W_QUALIFIER = 0.15 When "Score qualifier_keywords" is ON (default ON) W_QUALIFIER × (qual_matched / qual_total)
Role — title is the exact primary_role W_ROLE = 0.35 Always evaluated Flat: 0.35 if matched, else 0
Role — title is any other role_keywords variant W_ROLE_VARIANT = 0.25 Only when Tier 1 did NOT fire (mutually exclusive) Flat: 0.25 if matched, else 0
Recency W_RECENCY = 0.05 Always-on freshness preference Gauss decay on date_posted (offset 7d, scale 30d, decay 0.5)
Salary W_SALARY = 0.05 Only when min_salary is set Flat: 0.05 if comp meets the floor, else 0

Per-keyword weight inside a ratio bucket is bucket_weight / count, so each bucket spends its fixed budget no matter how long the list is. The role tiers are mutually exclusive; per-bucket misses just lower the ceiling — since every result in one query misses the same bucket, ranking within that query is unaffected.

How each field is searched + the scoring weights (v3.1 COT-modular)

v3.1 reshapes the role gate around a two-branch nested role_keywords and drops disambiguating_context entirely — routing-side disambiguation is now done by context_terms as a HARD GATE on the generic branch (strictly more powerful than the old soft boost). The role-tier ranking splits into three mutually-exclusive buckets; everything else is borrowed verbatim from v2.

Field → document-field mapping

Criteria field Role in the query Document fields hit
primary_role Scoring only (Tier 1) title.keyword (word-bounded regex)
role_keywords.direct_roles Hard gate (Branch 1) + scoring (Tier 2) title.keyword (word-bounded regex) — title only, one match is enough to qualify
role_keywords.generic_roles.roles Hard gate (Branch 2, AND with context_terms) + scoring (Tier 3) title.keyword (word-bounded regex) — title only; ALSO requires ≥1 context_term anywhere in the JD
role_keywords.generic_roles.context_terms Hard gate conjunct for Branch 2 (scoring: not a title term, doesn't rank on its own) title, description.full, description.about_company (match / match_phrase)
skill_keywords Hard gate + ratio scoring (always full 0.60 — no context bucket split) title, description.full, description.requirements, description.qualifications
qualifier_keywords Hard gate (+ optional ratio scoring via the toggle, default ON) title, description.full, description.about_company
anti_keywords Exclusion must_not on title
company_names, locations, workplace_type, job_type, visa_*, posted_within_days, min/max_experience_years Hard filters Same as v2 — equality / range clauses on the corresponding indexed fields
min_salary, max_salary SOFT scoring only — never a hard filter compensation.min (gte) / compensation.max (lte), plus optional currency / interval
Recency (always-on decay) Soft scoring, tightened from v2 date_posted gauss decay — offset 0d, scale 14d, decay 0.5 (half-life 14d, no plateau)

Scoring weights (sum ≈ 1.0)

Bucket Weight When it fires Formula
Skills (ratio) W_SKILLS = 0.60 Always, when any skill_keywords is supplied (no context-bucket split in v3.1 → always full 0.60) W_SKILLS × (skills_matched / skills_total)
Qualifier (ratio) W_QUALIFIER = 0.15 When "score qualifier_keywords" is ON (default ON) W_QUALIFIER × (qual_matched / qual_total)
Role — title is the exact primary_role W_ROLE = 0.35 Always evaluated (Tier 1) Flat: 0.35 if matched, else 0
Role — title is a direct_roles entry W_DIRECT_ROLE = 0.30 Only when Tier 1 did NOT fire (Tier 2 — mutually exclusive with primary_role) Flat: 0.30 if matched, else 0
Role — title is a generic_roles.roles entry W_GENERIC_ROLE = 0.25 Only when Tier 1 + Tier 2 did NOT fire (Tier 3 — mutually exclusive with primary_role AND direct_roles) Flat: 0.25 if matched, else 0
Recency W_RECENCY = 0.05 Always-on freshness preference (tightened from v2's 7d/30d plateau) Gauss decay on date_posted (offset 0d, scale 14d, decay 0.5)
Salary W_SALARY = 0.05 When min_salary and/or max_salary is set Flat: 0.05 if comp satisfies BOTH bounds (when both supplied), else 0

The three role tiers are mutually exclusive — a doc earns AT MOST one role bucket. Per-keyword weight inside a ratio bucket is bucket_weight / count; each bucket spends its fixed budget regardless of list length. Per-bucket misses lower the ceiling but don't affect within-query ranking (every result misses the same bucket).

— no query yet
Pipeline Per query Per day Per month (×30) Breakdown

Cost assumptions (click to edit)

Numbers below feed every row. Edit any field and the table re-renders. Defaults are public list prices in us-east-1 verified against: AOSS, Bedrock, OpenAI.

Per-query LLM token pricing

Vector-pipeline fixed infrastructure (only the NEW AOSS collection — existing infra is sunk cost)

Excluded (assumed sunk / equal across both pipelines): Glue ETL, S3 vendor storage, DynamoDB manifest, Lambdas + Step Functions, the existing tejas-jobs-vector collection. Real prod monthly is ~$150-300 higher than what's shown here, but both pipelines pay it.

Extracted criteria from NL query — no query yet
Raw LLM output (vector_filters.yaml) 
(no query yet)
mode offset d scale d decay
How each filter shapes the query (v7 vector pipelines)
Hard filters — exclude non-matching docs from the kNN candidate pool
FilterMechanismOpenSearch shape
city HARD FILTER on location.city + fallback match_phrase on location.raw {"bool": {"should": [{"terms": {"location.city": ["bangalore","bengaluru"]}}, {"match_phrase": {"location.raw": "Bangalore"}}]}}
state HARD FILTER on location.state {"term": {"location.state": "Karnataka"}}
country HARD FILTER on location.country + AUTO-ROUTE trigger (≠ India → switches to global CoT pipeline) {"term": {"location.country": "India"}}
seniority_band HARD FILTER on seniority_level OR seniority_levels_alt {"bool": {"should": [{"terms": {"seniority_level": ["junior","entry","intern"]}}, …]}}
company_name HARD FILTER via match_phrase on company_name {"match_phrase": {"company_name": "Stripe"}}
posted_within_days HARD FILTER on date_posted range {"range": {"date_posted": {"gte": "2026-06-01T..."}}}
Soft signals — reorder within the kNN top-K (no exclusion)
FilterMechanismWeight
skill_keywords SOFT BOOST per-term match_phrase on description.summary ×1.40 (max)
domain_keywords EMBED ENRICHMENT appended to query string before Bedrock embedding (v7-10k pill only — the "no domain embed" pill skips this)
+ SOFT BOOST per-term match_phrase on description.summary (both pills) 		shifts vector
×1.40 (max)
workplace_type SOFT BOOST per-mode term on workplace_type field ×1.40 (max)
experience_years_min / _max SOFT BOOST when experience.min_years ≤ user_max AND experience.max_years ≥ user_min ×1.40
(raw query) EMBEDDING — full text → Bedrock Titan v2 → 1024-d vector → kNN cosine against description.summary vectors base score [0, 1]
boost_company_followers toggle POST-RANK oversample top-K, join companies index, multiply by 1 + 0.005·log1p(followers), re-sort, slice page (v7-10k only) ×1.0 → ×1.08 (max at 10M followers)
Boost layers nest as four function_score wrappers (skill → domain → workplace → experience) with boost_mode: multiply, so a doc matching all four dimensions compounds to ~3.84×. Then the optional followers re-rank can lift another ~1.08× on top. Final _score is normalized to [0, 1] by dividing by the ceiling of currently-active multipliers.
Test combinations — mix and match these to verify behaviour
QueryWhat it tests
skill_keywords only (no hard filter)
city hard filter alone
workplace boost + experience boost + skill embed
4 hard filters + domain enrichment + skill boost
seniority hard + domain expansion (8 synonyms appended to embed)
country ≠ India → auto-route to global CoT
company_name hard filter
role + domain enrichment + city hard filter
Same query, two pills: compare on v7 (10k filter test) vs v7 10k (no domain embed) — different kNN candidate pools because the synonym tail is only on one
Query type examples

Vendor-feed pipeline

Architecture + monthly cost estimate. Hourly poller → archive → Glue ETL → OpenSearch. Numbers update from live manifest data.

Architecture

Side-channels: DynamoDB (manifest) ← archive/glue write status. SNS (alerts) ← failures only. CloudWatch (logs) ← all components.

Component cost breakdown

Component Unit price Estimated usage / mo Cost / mo
Estimates use AWS list prices (us-east-1) and the observed ingest rate from the manifest table. AOSS OCU baseline is shared with prod jobfetcher and is NOT included here.

Custom Run

Live Scraper Status

No active run

Live Log

Run History

ID Started Duration Status Trigger Scrapers Found New Dupes Expired Errors
Loading...
Time Scraper Level Company/URL HTTP Message Retries Resolved Actions
Loading...

Job flow

row 1 = today · row 2 = last 7 days · chart = last 50 ingest files (≈ hourly)
New jobs—
Already indexed—
Expired—
New jobs · this week—
Already indexed · this week—
Expired · this week—
LLM tokens · today—
Cost · today—
Cost so far—
■ New ■ Already indexed ■ Expired

New jobs / day

Detailed breakdowns

Top countries

Top cities

Applicant buckets

Compensation (min, USD)

Experience required (years)

Education level

Remote vs onsite

URL coverage

Per-file ingest (last 50 files)

Filename Status Data window Completed Records (total / new / refreshed / closed / errors) Bytes

Data Completeness by Source

Source Total Description Location Compensation Classification Other
Full Summary Resp. Req. Qual. Benefits About Co. City State Country Raw Min/Max Currency Interval Equity Job Type Role Cat. Seniority Dept Team Workplace Skills Exp. Education Visa Apply URL Date Posted
Loading...
Load Preset
Search Criteria
Generated JSON 

                            


                            

                                
curl

                                

                                    

                                    
                                

                            

                        

                        

                            

                                
                                
                                
                                

                                

                                    
                                    
                                

                                
                            

                            
                            
Fill in the form and hit Send

                            

                        

                    

                


                

                    

                


                

                    

                        Browse and edit the XML role and domain modules
                        under
                        prompts/nl/v3/roles-xml/ and
                        prompts/nl/v3/domain-xml/. These
                        feed the classifier and the v3 / COT-modular
                        pipelines. Saves write to disk atomically; the
                        in-memory module cache is invalidated so the
                        next request picks up your edit without a
                        server restart.
                    

                    

                        

                        

                            

                                Pick a module on the left, or click
                                + New to create one.
                            

                            
                        

                    


                    

                        

                            
New role