About This Project
How we collect, process, and present public economic data
The Daily Pulse is a free, open-source dashboard that aggregates publicly available U.S. economic data and presents it in an accessible, visual format. Our goal is to make government data more approachable — no paywalls, no subscriptions, no login required.
Every data point comes from official government sources (FRED, BLS, state labor departments) or established public APIs (HackerNews). We don't generate, estimate, or editorialize the numbers — we simply collect, organize, and visualize them.
The entire project is open source and available on GitHub.
Data Sources
All data comes from free, public sources. No paid APIs or proprietary datasets are used.
Federal Reserve Bank of St. Louis
Gas prices, unemployment rate, JOLTS job openings/hires/quits/layoffs, federal funds rate, 30-year mortgage rate, consumer sentiment, median income, CPI, rent index, personal savings rate.
U.S. Department of Labor
Consumer Price Index (CPI) across 15 categories: All Items, Food, Housing, Apparel, Transportation, Medical Care, Recreation, Education, Gasoline, Rent, and more.
Y Combinator / Algolia
Top and recent technology stories, keyword trends (AI, Rust, Python, layoffs, remote work, etc.), story scores, and comment counts.
State Departments of Labor
Worker Adjustment and Retraining Notification (WARN) Act layoff filings including company name, location, number of affected workers, and layoff dates.
Methodology
Automated scrapers fetch data from government APIs and public sources on a scheduled basis. FRED and BLS data is pulled daily, HackerNews hourly, and WARN filings weekly. All raw data is stored in a local SQLite database with full history.
Internet sentiment scores use the VADER (Valence Aware Dictionary and sEntiment Reasoner) algorithm, a rule-based model specifically tuned for social media text. Each post gets a compound score from -1 (most negative) to +1 (most positive), normalized to a 0-100 scale for display.
The Signal page computes Pearson correlation coefficients between different data series (e.g., Reddit layoff mentions vs. actual WARN filings). These correlations are recomputed daily and include time-lag analysis to identify leading indicators.
CPI and economic metrics show year-over-year (YoY) percentage changes calculated by comparing the latest value to the value from the same period 12 months prior. This removes seasonal effects and shows the real trend.
The purchasing power chart shows what $100 from 5 years ago would buy today, calculated as (base_CPI / current_CPI) x 100 using the CPI All Items index (CUSR0000SA0).
A custom composite score (0-100) combining the latest CPI values for Rent, Food, and Gas prices. Higher values indicate greater cost pressure on essential spending. The trend direction is determined by comparing the current month to the prior month.
- This dashboard is for informational purposes only. It is not financial or investment advice.
- Government data sources may have reporting delays of days to weeks. BLS CPI data is typically published with a 2-3 week lag.
- WARN Act data currently covers Texas only. Additional states will be added as reliable data sources are confirmed.
- Sentiment analysis (VADER) is rule-based and may not capture sarcasm, irony, or domain-specific nuance.
- Cross-correlations show statistical relationships, not causation. A high correlation coefficient does not imply one metric causes another.
- Historical data coverage varies by source. FRED data goes back to 2020; WARN filings to 2022; HN stories are rolling 90-day.