Skip to main content
✨ Run your entire business in one platform — CRM, HR, Accounting, Projects & more. Start Free Trial →

Turning World Bank Data Into 50K+ Searchable Pages with WordPress

Turning World Bank Data Into 50K+ Searchable Pages with WordPress
By: Dev.to Top Posted On: March 24, 2026 View: 1
What if you could make decades of World Bank and IMF economic data actually accessible and browsable - not buried in spreadsheets and PDF reports that nobody reads? That's what we built with historysaid.com: a programmatic SEO site that transforms raw international development data into 50,000+ structured, searchable pages. Every country, every indicator, every year - all queryable, all browsable, all indexed by Google. This post covers the full technical architecture: the data pipeline, the database design, the rendering engine, and the lessons we learned building it. The Data Problem The World Bank and IMF publish some of the richest economic datasets on the planet: GDP, inflation, trade balances, debt levels for 200+ countries Time series spanning 60+ years (some indicators go back to the 1960s) Hundreds of unique economic indicators covering everything from agricultural output to internet penetration rates Regular updates as new data gets published quarterly or annually But the official portals are designed for researchers and economists who already know what they're looking for. You need to know the indicator code (like NY.GDP.MKTP.CD for GDP in current USD) and use their clunky query builders to extract data into spreadsheets. There's no way to just... explore. To browse. To stumble upon interesting economic stories by clicking around. We wanted to change that. Why Not Just Build a Dashboard? We considered building a single-page dashboard app with interactive charts and filters. But dashboards have a fundamental SEO problem: they're one URL. Google can't index the state of your filters. If someone searches "Turkey GDP growth history", a dashboard app won't rank because that specific view doesn't have its own URL. Programmatic SEO solves this. Each unique combination of country + indicator gets its own page, its own URL, its own title, and its own meta description. Google can index all 50K of them. We chose WordPress for the same reasons we used it for startup-cost.com (see our previous post): cheap hosting, familiar ecosystem, and a powerful rewrite engine that nobody uses to its full potential. The Architecture We built hs-engine, a WordPress plugin that handles everything from data ingestion to page rendering. Data Pipeline The data flows through several stages before it becomes a browsable page: World Bank API --> JSON --> Parse & Validate --> Normalize --> MySQL IMF Data Portal --> CSV/JSON --> Parse & Validate --> Normalize --> MySQL | MySQL --> Virtual URL routing --> PHP Template --> Rendered HTML page Each data source has its own ingestion script because the formats differ significantly: // World Bank API ingestion (simplified) function hs_ingest_worldbank($indicator_code) $url = "https://api.worldbank.org/v2/country/all/indicator/$indicator_code?format=json&per_page=10000"; $pages = hs_fetch_all_pages($url); foreach ($pages as $entry) if ($entry['value'] === null) continue; // Skip null datapoints $country_id = hs_get_or_create_country($entry['country']['id'], $entry['country']['value']); global $wpdb; $wpdb->replace('hs_datapoints', [ 'country_id' => $country_id, 'indicator_id' => hs_get_indicator_id($indicator_code), 'year' => (int) $entry['date'], 'value' => (float) $entry['value'], ]); // IMF data ingestion (simplified) function hs_ingest_imf($dataset_code) $url = "https://www.imf.org/external/datamapper/api/v1/$dataset_code"; $data = json_decode(file_get_contents($url), true); foreach ($data['values'][$dataset_code] as $country_code => $years) $country_id = hs_get_or_create_country($country_code); foreach ($years as $year => $value) global $wpdb; $wpdb->replace('hs_datapoints', [ 'country_id' => $country_id, 'indicator_id' => hs_get_indicator_id($dataset_code, 'imf'), 'year' => (int) $year, 'value' => (float) $value, ]); The pipeline runs on a weekly WordPress cron job. When new data is published by the World Bank or IMF, our next scheduled run picks it up automatically. Database Schema The database design is simple but carefully indexed for our access patterns: CREATE TABLE hs_countries ( id INT AUTO_INCREMENT PRIMARY KEY, code CHAR(3) NOT NULL, -- ISO 3166-1 alpha-3 code_2 CHAR(2), -- ISO 3166-1 alpha-2 name VARCHAR(200) NOT NULL, slug VARCHAR(200) UNIQUE NOT NULL, region VARCHAR(100), income_group VARCHAR(50), population BIGINT, INDEX idx_code (code), INDEX idx_slug (slug) ); CREATE TABLE hs_indicators ( id INT AUTO_INCREMENT PRIMAR
Share:

Tags:
#0 

Read this on Dev.to Top Header Banner

Want to run a more efficient business?

Mewayz gives you CRM, HR, Accounting, Projects & eCommerce — all in one workspace. 14-day free trial, no credit card needed.

Try Mewayz Free →

Comments

Power your business with Mewayz ERP

All-in-one platform: CRM, HR, Accounting, Project Management, eCommerce & more. 14-day free trial.

Start Your Free Trial →

No credit card required · Cancel anytime · 131+ modules

Contact Us
Follow Us
Site Map
Get Site Map
About

Mewayz News brings you the latest breaking news, in-depth analysis, and trending stories from around the world. Covering politics, technology, business, sports, entertainment, and more — updated every hour, 24/7.

Mewayz Network

Mewayz App Stream Watch TV Music Games Tools Calculators Dictionary Books Quotes Recipes Photos Fonts Icons Study Papers Resume Templates Compare Reviews Weather Trading Docs Draw Paste Sign eBooks AI Learn Currency Convert Translate Search QR Code Timer Typing Colors Fitness Invoice Directory Social Seemless