The workflow before#
BlackRock's research analysts and portfolio teams gathered their own external data. Someone pulled the latest regulatory filing, someone else checked a company's investor relations page, and news and alternative data came in through whatever feed a team had set up. Every source was a manual job, so coverage stopped at about 50 sources and reached roughly 20% of what the teams said they needed. Analysts spent 30 to 40% of their week collecting and cleaning data instead of reading it. Information showed up daily or weekly, long after it was published, and each team kept its own copy, matched to securities in its own way.
What we built#
We built an automated collection system that reads the outside world for BlackRock's analysts. It crawls more than 1,000 sources, pulls out the companies, people and events in each document, and ties every mention to the right security in BlackRock's own security master. Analysts stop hunting for data and start with it already cleaned, linked and waiting in the warehouse they query every day.
Crawlers that cover 1,000+ sources
Filings from SEC EDGAR and international regulators, company sites, news and alternative data all flow in on their own schedule. Simple pages get a lightweight crawler, JavaScript-heavy sites get a full browser, and providers with an API get a direct client. We matched the collection method to each source instead of forcing one tool on all of them, which is what made 1,000+ sources practical to run.
Extraction that reads like an analyst
NLP models read each document, pull out companies, people, transactions and events, classify the document and score how relevant it is. An analyst covering a name sees the filing and the event it describes, not a raw page dump. Relevance scoring keeps the noise out of their queue.
Matching to the security master, with review
Every mention is resolved to BlackRock's security master by ISIN, CUSIP and ticker, and each match carries a confidence score. High-confidence matches go straight through. Low-confidence ones go to a human reviewer, because a mention attached to the wrong security is worse than one that's missing.
Alerts before data goes missing
Source sites change their layout without warning, and a broken crawler can fail quietly for weeks. Change detection flags when a site's structure shifts, so the team fixes the crawler before a gap reaches anyone's analysis. Freshness and completeness are tracked by source, so data engineers know where to look first.
How we delivered it
We started with the sources analysts used most and the security master match, since that's what made the data usable on day one. Coverage then grew in waves by source type: regulatory, company, news and alternative. Clean data landed in the S3 data lake and Snowflake that analysts, data scientists and the Aladdin team already used, so nobody had to learn a new tool to get value from it. The review queue gave research staff a direct say in match quality from the start.
Before and after#
| Measure | Before | After |
|---|---|---|
| Sources covered | 50, collected by hand | 1,000+, automated |
| Analyst time on data collection | 30 to 40% | Under 5% |
| Coverage of needed sources | About 20% | 95% |
| Data freshness | Daily or weekly | Near real time |
| Security matching | Done by each team its own way | One match to the security master, with a confidence score |
What's next#
With 1,000+ sources already resolved to the security master, the next step is agents that watch holdings for material events. They'd read the new filing or headline, draft a short summary and route only the items that need an analyst's judgment, so research time goes to the calls that matter.
