Every source class ranked by new contactable entities per hour of work — not by row count. Status is against what this corpus already holds: 186,461,039 rows / 8.1 GB of Parquet across 1,356 shards.
Rows ≠ entities ≠ leads. A business appears in a registry, a licence file, a tax permit and three permit sets — six rows, one lead. Everything below is judged on new entities and contact density, never on volume.
Nearly half the corpus is still unclassified by family — that bucket is wave-6 files whose slugs don't say what they are, and classifying it is a prerequisite for knowing what we can already sell. Of the 779 queued sources / ~23.3 M rows, 765 are ArcGIS and the biggest are permits — buying signals attached to businesses we already hold, not new entities.
| Source class | Access | Entities | Contact density | Volume available | Status | Why / next move |
|---|---|---|---|---|---|---|
| Occupational & professional licensing state boards |
Socrata / bulk | both | 46.9 M held ~32 M more |
have gap | 148 of 178 inventoried sources don't match the corpus. Big missing: TX insurance agents 4.42 M, CT licences 2.66 M, WA providers 2.45 M, CO 1.61 M. Licence numbers fill the empty identifiers table. | |
| Sales / use tax permit holders state dept of revenue |
bulk / API | business | 10–30 M est. | untapped | Best underused class. Only businesses with actual taxable revenue must register — filters out shell entities. CA CDTFA, TX Comptroller, WA DOR publish lists. | |
| OpenStreetMap POIs shop / amenity / office / craft |
Geofabrik bulk | business | 1–15 M US | untapped | Highest contact density of any free source — phone, website, street address, hours as first-class tags. Nothing of this class is in the corpus. One 11 GB download. | |
| SAM.gov entity registrations | API + key | business | ~1 M+ | needs key | POC name + postal address only. Email/phone are FOUO → federal-only. Check the bulk extract before paging a 10 k/day API. | |
| Federal award / vendor data USAspending · FPDS · Grants.gov |
API / bulk | business | millions of awards | untapped | Entities proven to win paid work, with dollar amounts — a qualification signal nothing else gives. | |
| IRS Business Master File tax-exempt orgs |
bulk CSV | business | ~1.9 M | partial | Nonprofits and foundations with addresses; partial coverage already held via wave 6. | |
| Commercial property & assessor rolls county parcel + deed |
ArcGIS / portal | both | county × 3,000 | untapped | Owner-of-record names and mailing addresses — the landlord/property-holding segment, largely business entities. | |
| UCC filings secured lending |
state SOS portals | business | state × 50 | untapped | Filing means a lender underwrote them — an active, creditworthy filter. Names + addresses. | |
| Health & food inspection data | ArcGIS / Socrata | business | city/county × 1,000s | partial | Restaurants and food service with premises addresses; continuous publishing, so it stays fresh. | |
| Alcohol / cannabis / gaming licences | state ABC bulk | business | ~500 k+ | partial | TTB permittees partly held (1.01 M); state ABC lists add on-premise venues. | |
| Childcare / private school licensing | state + NCES | business | ~500 k | untapped | Licensed facilities with addresses and often director names; NCES PSS covers private K-12. | |
| Job postings hiring = spending |
Adzuna API · USAJobs | business | continuous | untapped | Strongest timing signal available — a company hiring this week is buying this month. | |
| Trade association & chamber directories | scrape | business | thousands of lists | untapped | Member lists publish contact details by design. Small volumes, excellent density, high-effort to collect. | |
| Trade-show exhibitor lists | scrape | business | event × 1,000s | untapped | Exhibitors are pre-qualified buyers with budget. Usually the densest contacts per row anywhere. | |
| Court · liens · bankruptcy · tax-delinquent | court portals / state | both | continuous | untapped | Distress signals; also a rare source for sole-proprietor and officer names. | |
| Franchise disclosure (FTC FDD) | portal / scrape | business | franchise systems | untapped | Franchisors plus their unit counts; franchisees are high-spend, well-capitalised operators. | |
| Insurance producer / agency licences | state DOI | both | TX alone 4.42 M | gap | Already in the licensing inventory and unmatched — one of the largest single missing rosters. | |
| Website enrichment of rows we already hold | crawl | business | = existing rows | untapped | Cheapest path to the metric that matters: turn rows we already own into rows with an email or phone. Respect robots.txt; expect IP throttling. | |
| Fresh registration feeds | Socrata / RSS | business | ~39 k/month OR alone | untapped | A business registered this month needs everything. Highest intent, smallest volume — find the equivalents in each state. | |
| FCC ULS licensees + tower registrations | browser only | both | ~1.5 M | 403 to scripts | Ten files lost to bot-blocking. Needs a real browser session — not a retry. | |
| ATF federal firearms licensees | browser only | business | ~140 k | 403 to scripts | Same gate; small volume, complete coverage of the segment. | |
| California CSLB + DRE | browser only | both | ~280 k + ~600 k | postback-gated | The largest single blocked roster — ~880 k contractors and agents in one state. | |
| FL Sunbiz · NJ · OH · VA · GA · HI · KY | browser / account | business | millions | gated | Login walls, WAFs and JS-only apps. Browser session, one source at a time. | |
| Google Places · SafeGraph · ZoomInfo · Data Axle | paid | both | — | paid | Listed for completeness only. Not recommended without explicit authorisation to spend. |
Contact density = how often a row carries a phone, email or website: high · medium · low.
identifiers table — which is what
makes cross-source dedupe possible at all.| Question | Why it blocks |
|---|---|
| Is the goal 330 M rows, or distinct contactable entities? | 330 M rows is reachable; 330 M US businesses is arithmetically impossible (~34–36 M exist). The plan optimises entities and cannot honestly chase the row number. |
| Public corpus or private bucket? | Decides whether we publish a business-only cut with contacts stripped, or pay ~$1–2/month for a private bucket. Publishing as-is would expose consumer complaint records and individual practitioners. |
| API key: personal SAM.gov account, or register LCHWM LLC as a federal entity? | A personal non-Federal account unlocks all Public data. Registering the LLC is a much heavier process and only pays off for contract bidding, not for data. |
| How much browser-babysitting is worth it? | FCC, ATF and California are hours of hands-on work each. Breadth over depth says skip them and take the Socrata/ArcGIS and licensing rosters first. |
| Authorise paid sources? | Currently treating every gated source as blocked. Nothing is purchased without an explicit yes. |
WHAT WE HOLD — 186,461,039 rows / 8.1 GB / 1,356 shards
unclassified (wave 6) 88,900,158 47.7%
occupational licensing 46,895,732 25.2%
arcgis / socrata sweep 30,520,090 16.4%
federal 15,580,374 8.4%
permits 2,435,553 1.3%
health practitioners 1,206,782 0.6%
derived (dentists) 922,350 0.5%
QUEUED (779 sources) ~23,280,927 mostly permits
FUNNEL 186.5M rows -> distinct entities UNKNOWN -> contactable UNKNOWN
real US businesses ~34-36M (330M would be ~97% of the population)
PRIORITY
1 scoreboard: normalize + measure distinct and contactable
2 missing licensing rosters (~32M rows)
3 OpenStreetMap POIs (highest free contact density, absent today)
4 sales/use tax permit holders + fresh registration feeds
5 website enrichment of rows already owned