Beyond Magnets: Other Pressure Points

Okay, so the reliance on China for rare earth magnets is the elephant in the room – and recent export controls confirm it’s an elephant that can be prodded for geopolitical leverage. It’s the most stark example of a critical dependency. But it’s not the only potential pressure point hidden within the wind turbine’s supply chain, even if others are less severe.

Think about gearboxes, essential for many turbine designs that aren’t direct-drive²⁵. These are complex, high-precision machines. While major European suppliers like ZF/Winergy, Moventas, and Renk exist, China is also a key manufacturing hub and home to major players like China High Speed Transmission (NGC)^33,39. The market for these specialized components is relatively concentrated globally, meaning diversifying away from existing players, including Chinese ones, isn’t necessarily easy³².

Then there are the large, specialized bearings needed for the main shaft rotation, blade pitch adjustment, and nacelle yaw movement^25,40. Specific sourcing data can be scarce, but these have been flagged as potential bottlenecks, particularly as turbines get bigger and for the growing US offshore market^23,38. While global bearing manufacturers exist, scaling up alternative production quickly for these massive, custom-designed parts could be challenging³⁸.

Finally, consider the power electronics and control systems – the turbine’s brain and nervous system²⁸. These rely heavily on the global semiconductor and electronics supply chain, which, as we know, is heavily concentrated in Asia, including China^28,41. Even if the final control units are assembled or integrated regionally, the essential chips and other electronic components often start their journey in East Asia. Rapidly shifting this underlying dependency is a complex, long-term proposition.

So, while the rare earth magnet situation represents an extreme case – a near-monopoly on a critical material with few easy substitutes – these other dependencies on gearboxes, bearings, and electronics add further layers of complexity and moderate risk to the wind turbine supply chain. They underscore that even beyond the REE bottleneck, building resilience requires looking deep into the sourcing of multiple specialized components. Still, the magnet issue remains in a class of its own due to that unique combination of extreme market concentration and the sheer difficulty of developing alternatives at scale.

Conclusion: Recalibrating the Green Dream

So, back to my ill-fated Ørsted investment. My initial thinking was simplistic: tariffs are noise, the green trend is real, buy the dip. What I missed was the plumbing behind the green trend, but how exposed that plumbing is to geopolitical friction. The “noise” of tariffs directly provoked retaliatory Chinese export controls on the very rare earths and magnets essential for wind turbines. The fragility isn’t theoretical; it’s being actively demonstrated.

The wind industry starkly illustrates this. The critical vulnerability lies in the permanent magnets, powered by REEs processed almost exclusively in China, particularly the heavy REEs now subject to export licensing. This isn’t just a supply chain concentration; it’s a geopolitical chokepoint being actively tightened. Diversifying away, as the US is attempting with DoD backing, is a monumental, multi-year task facing vast scale and technological hurdles, dwarfed by China’s current capacity.

The fact that these materials are dual-use – critical for both green energy (wind turbines, EVs) and advanced defense systems – raises the stakes immensely. The competition and potential restrictions impact national security and climate goals simultaneously.

Building independent European or Western capacity is proving incredibly difficult, as the recent Northvolt bankruptcy highlights⁴². If major industrial projects struggle even for batteries, the challenge of creating an independent mine-to-magnet supply chain for REEs outside of China – overcoming technical gaps and massive scale differences – seems even more daunting. Failure risks ceding control over the pace and cost of the energy transition, leaving climate ambitions hostage to the supply of essential components from a single, dominant source that has shown willingness to restrict access.

Achieving our climate goals requires more than targets and subsidies. It demands a hard-nosed focus on industrial strategy and supply chain security, recognizing that the materials underpinning the transition are now squarely in the geopolitical crosshairs. We need serious, sustained investment in diversifying REE processing and magnet production, fostering alternatives⁴³, and acknowledging that building resilience against these active pressures will be slow, expensive, and absolutely critical. Ignoring the reality of where essential parts come from, and the geopolitical levers attached to them, is a gamble the green transition cannot afford.

Post script

The information that I draw on in this article comes from Gemini 2.5’s deep reasearch function. I asked for information on concetrantion in the wind turbine industry, as well as a deep dive on manufacturing in the industry. The results were impressive, and I have included the two PDFs below.

Here is the deep dive on manufacturing in the wind turbine industry.

quadrantChart
    title Wind Turbine Component: China Dependency vs. Sourcing Difficulty
    x-axis Low China Dependency --> High China Dependency
    y-axis Many Sourcing Alternatives --> Few Sourcing Alternatives
    quadrant-1 Strategic Vulnerability
    quadrant-2 Specialized Tech
    quadrant-3 Resilient
    quadrant-4 Concentrated Mfg
    Blades: [0.6, 0.4]
    Towers: [0.2, 0.2]
    Nacelle Assembly: [0.61, 0.2]
    Gearbox: [0.7, 0.6]
    PMSG: [0.95, 0.95]
    Generator: [0.4, 0.2]
    Bearings: [0.72, 0.65]
    Power Electronics: [0.85, 0.75]

What tariffs?

On Wednesday, President Trump has announced tariffs on basically all of the United States of America’s trading partners as part of his “liberation day”, excluding Russia, North Korea, Cuba and Belarus. The EU received a 20% tariff, the UK a 10% tariff and Landlocked Lesotho received a 50% tariff on their $200m in exports to the US. Importantly, these tariffs were not imposed on goods which the US could feasibly produce domestically, but rather across the board on all goods, according to the trade deficit between the USA and the country in question.

However the highest effective rate is on goods from China. Following a sequence of escalations – an initial 10% tariff imposed February 1st, 2025, doubled to 20% on March 4th, and augmented by an additional 34% on April 4th – imports from China now face an effective tariff rate of 54%. This development, documented in timelines like this PIIE blog post, makes Chinese goods significantly more expensive in the US market. While tariffs have also been applied broadly, including to allies, the situation with USMCA partners appears more fluid. Recent events, such as the temporary delay of tariffs on Mexico following President Sheinbaum’s reported commitment to enhance border security, suggest tariffs might be employed as leverage for other political goals. Therefore, while uncertainty persists, there’s a working assumption here that deep North American integration (USMCA) might offer a more stable long-term sourcing advantage compared to other regions facing sustained high tariffs.

My idea is that US companies already manufacturing goods domestically and successfully exporting them might be able to redirect that output to the US market, filling the gap left by newly expensive imports. This is particularly relevant for product categories where the US both imports significantly (especially from China) and maintains some level of domestic production and export capability.

Mapping Trade Flows in a High-Tariff World

The USA imported $436B worth of goods from China in 2023, as shown in the treemap from the OEC below.

Examining US trade patterns through the Harmonized System (HS) codes highlights the key areas impacted. Based on recent annual trade data, major US imports from China include at the two digit HS code level:

• HS Chapter 85: Electrical machinery/equipment (electronics, components) - roughly $127 billion annually.
• HS Chapter 84: Machinery, mechanical appliances (industrial equipment, computer hardware) - around $85 billion.
• HS Chapter 95: Toys, games, and sports requisites - about $32 billion.
• HS Chapter 94: Furniture, bedding, and lighting - approximately $21 billion.
• HS Chapters 61 & 62: Apparel (knit and non-knit) - combined around $18.4 billion.
• HS Chapter 64: Footwear - roughly $10.3 billion.
• HS Chapter 87: Vehicles (largely parts) - about $18 billion.

While the US does export substantial amounts in some of these broad categories (like HS 84, 85, 87, and 90 - Medical/Optical Instruments), the products often differ. US exports frequently lean towards higher-value, specialized items, whereas imports from China often consist of mass-produced consumer goods or intermediate components. Acknowledging this, our analysis uses these broad categories to identify sectors where some potential for domestic substitution exists, recognizing this requires making broad assumptions to initiate the discussion.

Identifying Potential Relative Winners

In this challenging environment, the focus shifts to resilience and adaptation. Companies potentially better positioned are those with:

1. Existing, strong US or North American manufacturing operations.
2. Lower direct reliance on inputs now subject to the highest tariffs (especially from China).
3. Products falling into categories where the US has existing export capabilities, suggesting some domestic production know-how.

Sectors housing such companies might include:

• Medical Devices: Manufacturers where quality control favors regionalized or domestic production.
• Energy: Businesses focused on North American resource extraction and infrastructure.
• Agribusiness: Processors and equipment makers with strong domestic foundations.

The Complex Reality of Reshoring

This potential shift is far from simple. The very reason production moved offshore – lower costs – highlights the economic challenge tariffs attempt to override. Successfully pivoting production involves significant hurdles:

• Investment & Retooling: Shifting production lines, even for superficially “similar” goods within an HS code, requires capital investment and time. It’s not an overnight process.

• Skills Gap: As illustrated by the difficulties in domestically producing even seemingly simple items like a BBQ scrubber as discussed on a favorite podcast called Search Engine, specialized manufacturing skills (like tooling and die-making for plastics) may be scarce in the US, hindering rapid expansion.

• Labor-Intensive Goods: Industries like Apparel (HS 61/62) and Footwear (HS 64) rely heavily on low labor costs abroad. Reshoring these at a competitive price point is highly improbable. Consumers should anticipate significantly higher prices for these goods. This also creates hardship for workers in affected exporting countries.

• Irreplaceable Imports: Certain goods (e.g., coffee, bananas) cannot be produced at scale domestically due to climate, making tariffs purely inflationary for those items.

Context and Risks: Retaliation, Timeframes, and Diversification

The global trade environment under widespread tariffs is inherently confrontational. Retaliatory tariffs from China and other nations are a reality, directly impacting the existing export markets of US companies. This complicates the ability to pivot; companies might be forced to rely more on the domestic market not just out of opportunity, but because their export options are shrinking. In this “crazy situation,” the goal becomes identifying firms best equipped to minimize losses and adapt, rather than expecting windfall gains.

Furthermore, this industrial shift requires time. Given a potential four-year presidential term horizon, companies with less complex products or more adaptable production lines might be favored, as they could potentially complete a domestic pivot more quickly.

It’s also worth noting that this potential tariff-driven reshoring builds on existing trends. Companies had already begun diversifying supply chains away from China for several years due to factors like: tariffs imposed during Trump’s first term, heightened awareness of geopolitical and pandemic-related supply risks, and rising labor costs within China itself, making other locations relatively more attractive (as discussed in resources like this Money and Macro video). Companies that already initiated such diversification may have an advantage now.

Conclusion: Navigating Uncertainty

The current high-tariff environment presents a potential inflection point for US manufacturing. While significant opportunities could arise for domestic companies capable of capturing market share from expensive imports, the path is complex and uncertain. Success likely favors firms with strong North American operations, relevant product lines, and the agility to adapt production within a challenging multi-year timeframe, all while navigating retaliatory measures and domestic operational hurdles.

The persistence of these tariffs, the precise treatment of USMCA partners, and the practical ability of individual companies to execute a domestic pivot remain critical unknowns. Therefore, while this analysis identifies potential areas of interest, it underscores the need for deep, company-specific research and caution. This is just an economic framework for considering the potential industrial consequences of a dramatically altered trade policy.

Overview of HISCO

The Historical International Standard Classification of Occupations (HISCO) is a standardized system developed to classify and compare historical occupations across different countries and time periods. It is based on the International Standard Classification of Occupations (ISCO) from 1968 and has been adapted for historical research purposes. HISCO provides a common framework that enables researchers to analyze social structures, economic development, and labor market changes over time.

Structure of HISCO

HISCO employs a hierarchical coding system consisting of five digits:

• Major Groups: The first digit represents broad occupational categories.
• Minor Groups: The second digit specifies subgroups within the major group.
• Unit Groups: The third digit further refines the occupation category.
• Headings: The fourth and fifth digits provide detailed occupational titles.

This structure allows for detailed classification and analysis of occupations.

Related Classification Systems

HISCO serves as a foundation for other classification systems that analyze historical social stratification:

• HISCLASS: An occupational stratification scale that classifies occupations into social classes based on HISCO codes, facilitating international comparisons across different periods, countries, and languages.

• HISCAM: A social status scale derived from HISCO, developed to analyze social stratification and inequality in historical contexts. These systems provide additional tools for researchers studying historical social structures and mobility.

Resources and Tools

For further exploration and practical application of HISCO, consider the following resources:

• HISCO Database: This database offers access to occupational titles and their corresponding HISCO codes, as well as CSV downloads.

• R Package for HISCO Classification: An R package developed for classifying HISCO codes into social class systems such as HISCLASS, HISCLASS 5, SOCPO, and Universal HISCAM. This tool aids researchers in analyzing historical occupational data.

• Machine Learning Based Classifier This GitHub repo hosts “a tool designed to transform occupational descriptions into standardized HISCO (Historical International Standard Classification of Occupations) codes automatically. Developed by Christian Møller Dahl, Torben Johansen and Christian Vedel from the University of Southern Denmark, this tool leverages the power of a finetuned language model to process and classify occupational descriptions with high accuracy, precision, and recall.”

Conclusion

This post provides a structured overview of HISCO, its major groups, related classification systems, and available resources. By consolidating this information, it serves as a convenient reference for quickly accessing and understanding historical occupational classifications.

Prompt

Following Tobias Zwingmann’s guide to prompting Large Language Models, I provided the following prompt to GPT-4.5:

Role: you’re an expert on domestic solar energy systems.

Goal: I want you to conduct some research for me, and write an engineering report. I am trying to decide on a solar panel and battery system to install in my freestanding house in South Africa.

Tasks: There are a few considerations;

• I need to understand the size of the system I need to run everything electric in my house, except water heating and air conditioning,

• I need to understand what the lifespan of the system will be,

• Using this, I need to understand how long the payback period would be, considering the cost of installing the system, the savings on electricity while the system is in use, and the opportunity cost of that money invested in a high-yield savings account.

Details: Say that I live in a house that has a roof conducive to installing panels, that I pay a per kilowatt rate for my electricity from the grid and that I cannot sell back to the grid at the moment.

Response

Here is the response I received:

Streamlit Application

The model also built a Streamlit application to help me calculate the payback time of the system. You can access the application here.

There are quite a number of assumptions that I still need to work through…

What does it mean for teachers?

It’s worth pausing to consider what this means. Not long ago, it took me a long time to build tools like this from scratch—debugging code, tweaking visualizations, and ensuring everything worked smoothly for my students. Now, a single prompt can generate a similar interactive component in moments.

Clearly Claude can help educators create tools that help students learn with minimal effort. I wonder if the novelty of an interactive web app as a learning tool will rub off rather swiftly. Perhaps part of their appeal is that they are not the norm, and that educators put a lot of effort into creating them? I am not sure. But it is bittersweet to see that one of the parts of my job I enjoyed the most has been made so trivial as to require just one prompt. A friend of mine who works as an animator said same thing about image generating LLMs a few years ago, and it is really sinking in for me now, too.

The Code

If you’re interested in the code, here it is:

/**
 * CobbDouglasVisualizer Component
 * 
 * This React component visualizes the Cobb-Douglas production function,
 * allowing users to interactively adjust key parameters and observe 
 * how changes affect output.
 * 
 * The production function is given by:
 * 
 *    Y = A × K^α × N^(1-α)
 * 
 * where:
 * - A (Total Factor Productivity) represents overall efficiency.
 * - K (Capital) is the amount of capital input.
 * - N (Labor) is the amount of labor input.
 * - α (Output Elasticity of Capital) determines how much capital contributes to output.
 * 
 * Features:
 * - Users can adjust A, N, α, and K using range sliders.
 * - A line chart compares the current production function against a baseline.
 * - A bar chart compares output at a selected capital level (K).
 * - A dynamic tooltip provides economic insights on diminishing marginal returns.
 * - Uses Recharts for data visualization.
 * - Responsive design for optimal display on different devices.
 * 
 * Key Components:
 * - useState: Manages user-adjustable parameters and computed data.
 * - useEffect: Recalculates the production function curve when parameters change.
 * - Recharts:
 *   - LineChart: Displays the Cobb-Douglas function over different capital levels.
 *   - BarChart: Compares output at a fixed capital level.
 * - JSX-based UI with Tailwind classes for styling.
 * 
 * Usage:
 * - Import and include <CobbDouglasVisualizer /> in a React project.
 * - Adjust the sliders to see how changes in capital, labor, and productivity affect output.
 * - Observe diminishing marginal returns to capital in the visualized curves.
 */


import React, { useState, useEffect } from 'react';
import { LineChart, Line, BarChart, Bar, XAxis, YAxis, CartesianGrid, Tooltip, Legend, ResponsiveContainer } from 'recharts';

const CobbDouglasVisualizer = () => {
  // Default parameter values
  const [A, setA] = useState(10); // Total factor productivity
  const [N, setN] = useState(10); // Labor
  const [alpha, setAlpha] = useState(0.3); // Output elasticity of capital
  const [K, setK] = useState(10); // Current capital level
  const [data, setData] = useState([]);
  const [baselineData, setBaselineData] = useState([]); // Add baseline for comparison
  
  // Generate more granular data points for the production function
  useEffect(() => {
    const newData = [];
    const baseline = [];
    
    // Generate 100 points for a smooth curve from 1 to 20
    for (let k = 1; k <= 20; k += 0.2) {
      const roundedK = parseFloat(k.toFixed(1));
      // Production function with current parameters
      const output = A * Math.pow(roundedK, alpha) * Math.pow(N, (1 - alpha));
      
      // Baseline production function (with A=10, alpha=0.25, N=10)
      const baselineOutput = 10 * Math.pow(roundedK, 0.25) * Math.pow(10, 0.75);
      
      newData.push({
        k: roundedK,
        output: parseFloat(output.toFixed(2))
      });
      
      baseline.push({
        k: roundedK,
        output: parseFloat(baselineOutput.toFixed(2))
      });
    }
    
    setData(newData);
    setBaselineData(baseline);
  }, [A, N, alpha]);

  // Calculate output at current K
  const currentOutput = A * Math.pow(K, alpha) * Math.pow(N, (1 - alpha));
  const baselineOutput = 10 * Math.pow(K, 0.25) * Math.pow(10, 0.75);
  
  // Data for comparison bar chart
  const barData = [
    { name: 'Current', output: parseFloat(currentOutput.toFixed(2)) },
    { name: 'Baseline', output: parseFloat(baselineOutput.toFixed(2)) }
  ];

  return (
    <div className="p-6 max-w-4xl mx-auto bg-white rounded-lg shadow-lg">
      <h1 className="text-2xl font-bold mb-4 text-center">Cobb-Douglas Production Function</h1>
      <h2 className="text-xl mb-6 text-center">Y = A × K<sup>α</sup> × N<sup>(1-α)</sup></h2>
      
      <div className="grid grid-cols-1 md:grid-cols-2 gap-6 mb-8">
        <div className="p-4 bg-gray-50 rounded-md">
          <h3 className="text-lg font-semibold mb-4">Parameters</h3>
          <div className="space-y-4">
            <div>
              <label className="block text-sm font-medium text-gray-700 mb-1">
                Total Factor Productivity (A): {A.toFixed(2)}
              </label>
              <input
                type="range"
                min="1"
                max="20"
                step="0.5"
                value={A}
                onChange={(e) => setA(parseFloat(e.target.value))}
                className="w-full"
              />
            </div>
            <div>
              <label className="block text-sm font-medium text-gray-700 mb-1">
                Labor (N): {N.toFixed(2)}
              </label>
              <input
                type="range"
                min="1"
                max="20"
                step="0.5"
                value={N}
                onChange={(e) => setN(parseFloat(e.target.value))}
                className="w-full"
              />
            </div>
            <div>
              <label className="block text-sm font-medium text-gray-700 mb-1">
                Output Elasticity of Capital (α): {alpha.toFixed(2)}
              </label>
              <input
                type="range"
                min="0.01"
                max="0.99"
                step="0.01"
                value={alpha}
                onChange={(e) => setAlpha(parseFloat(e.target.value))}
                className="w-full"
              />
            </div>
            <div>
              <label className="block text-sm font-medium text-gray-700 mb-1">
                Level of Capital (K): {K.toFixed(2)}
              </label>
              <input
                type="range"
                min="1"
                max="20"
                step="0.5"
                value={K}
                onChange={(e) => setK(parseFloat(e.target.value))}
                className="w-full"
              />
            </div>
          </div>
          
          <div className="mt-6 p-4 bg-blue-50 rounded-md">
            <h4 className="font-medium text-blue-800">Output Comparison:</h4>
            <p className="font-bold">Current: Y = {currentOutput.toFixed(2)}</p>
            <p className="text-gray-600">Baseline: Y = {baselineOutput.toFixed(2)}</p>
            <p className="text-sm text-gray-600 mt-2">
              Current: A={A.toFixed(2)}, K={K.toFixed(2)}, N={N.toFixed(2)}, α={alpha.toFixed(2)}
            </p>
            <p className="text-sm text-gray-600">
              Baseline: A=10, K={K.toFixed(2)}, N=10, α=0.25
            </p>
          </div>
        </div>
        
        <div className="space-y-6">
          <div>
            <h3 className="text-lg font-semibold mb-2">Output Comparison at K = {K.toFixed(1)}</h3>
            <div className="h-40">
              <ResponsiveContainer width="100%" height="100%">
                <BarChart data={barData}>
                  <CartesianGrid strokeDasharray="3 3" />
                  <XAxis dataKey="name" />
                  <YAxis domain={[0, 'auto']} />
                  <Tooltip />
                  <Bar dataKey="output" name="Output" fill="#4ade80" />
                </BarChart>
              </ResponsiveContainer>
            </div>
          </div>
          
          <div>
            <h3 className="text-lg font-semibold mb-2">Production Functions</h3>
            <div className="h-64">
              <ResponsiveContainer width="100%" height="100%">
                <LineChart margin={{ top: 5, right: 30, left: 20, bottom: 20 }}>
                  <CartesianGrid strokeDasharray="3 3" />
                  <XAxis 
                    type="number"
                    dataKey="k" 
                    allowDataOverflow={true}
                    domain={[1, 20]}
                    label={{ value: 'Level of capital (K)', position: 'bottom', offset: 0 }}
                  />
                  <YAxis 
                    label={{ value: 'Level of output (Y)', angle: -90, position: 'insideLeft', offset: -5 }}
                    domain={[0, 'auto']}
                  />
                  <Tooltip />
                  <Legend />
                  
                  {/* Current Production Function */}
                  <Line 
                    data={data}
                    type="monotone" 
                    dataKey="output" 
                    stroke="#4ade80" 
                    strokeWidth={2}
                    dot={false}
                    name="New production function"
                    activeDot={{ r: 8 }}
                  />
                  
                  {/* Baseline Production Function */}
                  <Line 
                    data={baselineData}
                    type="monotone" 
                    dataKey="output" 
                    stroke="#f59e0b" 
                    strokeWidth={2}
                    dot={false}
                    name="Baseline"
                  />
                  
                  {/* Vertical line at current K */}
                  <Line 
                    data={[
                      { k: K, output: 0 },
                      { k: K, output: Math.max(currentOutput, baselineOutput) * 1.1 }
                    ]} 
                    type="monotone" 
                    stroke="#475569" 
                    strokeWidth={2}
                    strokeDasharray="5 5"
                    dot={false}
                    name="Current K"
                  />
                </LineChart>
              </ResponsiveContainer>
            </div>
          </div>
        </div>
      </div>
      
      <div className="bg-yellow-50 p-4 rounded-md">
        <h3 className="font-semibold text-yellow-800">Economics Insight</h3>
        <p className="mt-2 text-sm">
          The curvature of the Cobb-Douglas production function demonstrates diminishing marginal returns to capital:
          as more capital is added, the additional output gained from each unit decreases.
        </p>
        <p className="mt-2 text-sm">
          Parameter α directly controls this curvature:
        </p>
        <ul className="mt-1 text-sm list-disc pl-5">
          <li>When α is close to 0: The curve flattens quickly, showing rapid diminishing returns to capital</li>
          <li>When α is close to 1: The curve remains steeper, indicating slower diminishing returns</li>
          <li>Changes in A (productivity) shift the entire curve up or down</li>
          <li>Changes in N (labor) also shift the curve's position while maintaining its shape</li>
        </ul>
      </div>
    </div>
  );
};

export default CobbDouglasVisualizer;

The Spark of Inspiration

The inspiration came from a personal experience. My girlfriend Kathy lives in Paris, while I call Lund, Sweden home. Over the past couple of years, I’ve made many trips to Paris – mostly by plane (I do my best to minimize my environmental footprint in other ways). One recurring frustration was the transit experience at Paris CDG. Arriving in the evening often meant missing the last RER B train, leaving me to try and find a fellow frustrated flier to share an expensive taxi ride with into the city. By contrast, Copenhagen’s airport offers a seamless metro connection that runs round the clock from the airport into the city (the trains drive themselves!).

This contrast led me to wonder: how do transit options from European airports compare? How do factors like travel time, cost, and service frequency vary from one city to another?

The Challenge

My goal was clear, but I didn’t want to have to walk through each step myself, wading through blogs, official websites and Google Maps to collect the data. I wanted to focus on the insights, not the grunt work. I had in mind a two-step process:

1. Data Collection:
   • Start with the list of Europe’s busiest airports from Wikipedia.
   • For each airport, gather details on the distance to the city center and the transit options available—covering travel time, frequency, and fare estimates for modes like trains, buses, taxis, and metros.
2. Interactive Visualization:
   • Create a dashboard with an interactive scatterplot and a clickable map for users to compare transit options.
   • Include charts that, for example, highlight the biggest fare differences or the taxi price per kilometer by airport.

I pictured a simple app to peak at the data, and perhaps a scatter plot that I could use to compare different metrics.

Testing AI Capabilities

Back when OpenAI’s in March of 2023, just after signing up for ChatGPT Plus, I set it the task. I wanted a solution that could handle the tedious parts—googling, scraping, and assembling a consistent dataset—so I could focus on the insights. Unfortunately, GPT-3.5 struggled; its inability to search the web and inconsistent outputs meant that my vision remained out of reach.

I started down the path of building a pipeline that used a search engine API to find the data, pass it to GPT-3.5 for processing, and asked the LLM to produce the structred data for each airport. Prior to the support for structured output, the JSON that I got back was hit and miss both in terms of content and structure. This required a lot of manual intervention.

I then tried a few other models, including Claude Sonnet 3.5, which did a reasonable job but still required more intervention than I would have liked. This weekend I asked OpenAI’s o3-mini-high model to give it a go. I gave it a clear prompt:

“I want to create a dataset of transit information for the 50 busiest airports in Europe to the city centers they serve. Each result should include the distance in kilometers, and for each transit method—such as taxis, trains, trams, or buses—provide travel time, frequency, and one-way fare in euros, all formatted in a consistent JSON schema.”

To my surprise, the model succeeded for most airports, delivering data for 44 out of 50 – a decent starting point.

Here is an example of the JSON output for Heathrow Airport in London:

    {
      "name": "Heathrow Airport",
      "city": "London",
      "distance_km": 24,
      "airport_coords": { "lat": 51.4700, "lon": -0.4543 },
      "city_coords": { "lat": 51.5074, "lon": -0.1278 },
      "transit_options": [
        {
          "mode": "Heathrow Express",
          "transit_type": "train",
          "travel_time_minutes": 20,
          "frequency": "every 15 minutes",
          "fare_euros": 25
        },
        {
          "mode": "Taxi",
          "transit_type": "taxi",
          "travel_time_minutes": 50,
          "frequency": "on-demand",
          "fare_euros": 65
        },
        {
          "mode": "Bus",
          "transit_type": "bus",
          "travel_time_minutes": 55,
          "frequency": "every 30 minutes",
          "fare_euros": 8
        }
      ]
    },

Insights & Opportunities

The app immediately uncovered some interesting patterns:

• Transit Variation:
  Unsurprisingly, when looking at travel time and holding distance constant, we see that the fastest transit options are trains and metros (dark blue and pink), while taxis are next fastest (light blue), followed by busses (in red).

• Fare Differentials:
  Busses and metros lead the pack in affordability, with trains and taxis costing more. The scatterplot reveals that some airports—like Geneva and Zurich—have a wide range of fares, while others—like Istanbul and Madrid—offer more consistent pricing.

I think that the current app is a very good start, especially given that it took me a couple of hours to refine based on the output from the o3 model.

Looking ahead, there are plenty of opportunities to improve the app:

• Population Metrics:
  Incorporating data on city populations might explain why airports in mega-cities like Paris and London have longer distances and varied transit options.

• Data Quality:
  A more robust quality assurance process could update outdated figures—such as revised taxi fares—to ensure the app remains current. I picked up some old data for the RER B train from CDG to Paris, for example, which now costs €12, rather than the €10 I have in the dataset.

Conclusion

Transit Trek is just a dashboard. With the OpenAI o3-mini-high model, I could automate a process that once required a great deal of manual effort. I think this was a useful benchmark for me to understand the capabilities of modern LLM systems and how they can support our work as idea havers.

Conclusion

In conclusion, Kathy was right: grey dominates the couch color spectrum on NeverTooSmall.

I hope you enjoyed following along on this journey to answer a casual question with computer vision. If you’d like to try it yourself, feel free to check out the GitHub repo here and let me know what you find!

Our Approach

The concept is simple: think StackOverflow, but with a focus on SCB data. Researchers ask, answer, and, in the process, craft a self-sustaining knowledge base.

We have built an alpha version and are looking to get a small amount of money for a custom domain and hosting costs.

The current version of the site is accessible here. I include a screenshot of the homepage below.

Technical Underpinnings

We chose Google Firebase Firestore for its simple storage and Firebase for authentication. The upshot? A secure, efficient backend without the headache of password management or the risk of exposing passwords inadvertently. There is also a generous free tier offered for storage on Firebase.

The Firestore nosql database has a simple set up with a collection for users with a unique id, usernames and emails. The questions collection has a unique id for each question, the information pertaining to each question like title and body and date, as well as a subcollection for the number of upvotes and id of the user who upvoted to prevent multiple upvotes from the same user. Each answer to the question belongs to a sub collection of the question called ‘answers’ in the nosql database.

Frontend Innovation

SvelteKit powers our frontend; its dynamic nature is perfect for crafting reactive user interfaces. SvelteFire slots let us tailor content based on user status, enhancing interaction without clutter.

For example, compare the below in which logged in users can upvote questions and answers and add the their own answers with the view of logged out users who can just view the content.

User-Centric Design

The interface is intuitive. If you’re logged in, you’re part of the conversation. If not, you’re still in the know.

Deployment Strategy

Netlify hosts our site, chosen for its scalability and future-proofing abilities. It’s a platform that grows with us, promising smooth custom domain integration down the line.

Next Steps

Markdown rendering is next on our list. It’s about enriching the discourse with equations, links, and well-formatted content.

A Note on Development

Our secret weapon during development? ChatGPT-4 vision. It’s a tireless coding partner, turning visual and textual prompts into Svelte components that click into place. Iterating is swift.

Contribute

The project is public on GitHub in this repo. Feel free to add a comment in the discussions tab with a suggestion about how we can improve before launch.

Data

For the purpose of this tutorial, I have generated 20 place names to geocode and a column of arbitrary values for visualization.

R and tidygeocoder

Jesse Cambon has kindly made a package called tidygeocoder available on github.

You can install it on your computer using the following code:

install.packages('tidygeocoder')

Here is how you geocode a list of addresses using the tidygeocoder package.

Note that lat and long here will be the column names for the coordinates that are returned.

The line method = "osm" specifies that we are using the Open Street Map Nominatim API. Here you can switch to another engine if you prefer.

library(tidyverse) # for data manipulation
library(tidygeocoder) # for geocoding

# Read in data
library(readxl) # for reading excel files
places <- read_excel("data/data_to_geocode.xlsx")

# Geocode places
places_geocoded <- places %>%
    geocode(place_name,
        method = "osm",
        lat = latitude,
        long = longitude
    )

This what the output looks like when the geocoding is complete.

Passing 20 addresses to the Nominatim single address geocoder [===================================] 20/20 (100%) Elapsed: 20s Remaining: 0s

We can visualize the results using the ggplot2 package.

# Find bounds to trim map, adding some padding
bounds <- places_geocoded %>%
    summarise(xmin = min(longitude) - 0.1,
              xmax = max(longitude) + 0.1,
              ymin = min(latitude) - 0.1,
              ymax = max(latitude) + 0.1)

# Plot places
ggplot(places_geocoded, aes(longitude, latitude), color = "grey99") +
    borders() +
    geom_point(aes(size = arbitrary_value), colour = "midnightblue") +
    ggrepel::geom_label_repel(aes(label = place_name)) +
    theme_void() +
    coord_cartesian(xlim = c(bounds$xmin, bounds$xmax), 
                    ylim = c(bounds$ymin, bounds$ymax)) +
    labs(size = "Arbitrary value")

Great! We have successfully geocoded our list of addresses.

Geocoding in python

I have created a Colab notebook that shows you how to geocode addresses using the Google Maps Geocoding API in python. You can simply run the code in the notebook to get the coordinates of your own data by providing your own API key and data.

You can have a look at the Colab notebook here

Real world use case

In reality, it will likely not be as simple as running the above code once and getting all the coordinates you need.

You will likely need to do multiple passes, modify addresses, and evaluate the results.

Multiple passes are necessary to maximize the accuracy and success rate of geocoding. The initial pass may not successfully geocode all addresses due to inaccuracies, ambiguities, or limitations of the geocoding tool. By modifying the problematic addresses and doing subsequent passes, you can resolve these issues and geocode more addresses correctly.

For example, an address might fail to geocode because it contains a typo, is too specific, or too ambiguous. By modifying the address to correct the typo or by making it more general or more specific, it might geocode successfully in a subsequent pass.

The term “out of bounds” in the context of geocoding refers to the geocoded results that do not fit within a predefined geographical area or are not reasonable based on prior knowledge. For instance, if you are geocoding addresses in Sweden, any result that lies outside the borders of Sweden would be considered “out of bounds.”

In practical terms, if you know the general area where the addresses should be located (like a specific country or city), you can compare the geocoded results against this area. If the geocoded location falls outside this area, it is considered out of bounds, and you might need to modify the address and try geocoding it again, or use a different geocoding engine that might provide more accurate results.

Here is a diagram that shows the real world process of geocoding in more detail.

stateDiagram-v2
    [*] --> FirstPass
    FirstPass --> StoreResults: In bounds
    FirstPass --> ModifyAddresses: Out of bounds
    
    ModifyAddresses --> SecondPass
    
    SecondPass --> StoreSecondPassResults: In bounds
    SecondPass --> Evaluate: Out of bounds
    
    StoreResults --> Evaluate
    StoreSecondPassResults --> Evaluate
    
    Evaluate --> ManualGeocode: Handful left
    Evaluate --> TryDifferentEngine: Many left
    
    ManualGeocode --> FinalResults
    TryDifferentEngine --> FinalResults
    
    FinalResults --> [*]

An example: what was Consumer Price Inflation in 2022?

Say for instance that we want to know what Consumer Price Inflation was in 2022 in South Africa. We can ask the question in a natural language query, and the system will return the answer. Here is the query and the answer, along with the source documents that the system used to answer the question:

In the table above, we can see two of the chunks of text that are most similar in terms of content to our query, from the ChromaDB database that contains the 2023 Budget Review and Budget Speech broken into 2000 token chunks. The first chunk in our table is from page 39 of the budget review, and the second chunk is from page 27. Both of these chunks contain the information that we are looking for, and the system has correctly identified them as the most relevant chunks.

These chunks are then passed to the OpenAI GPT-3.5 large language model, which is able to take in the question and the context, and then return to us the answer. In this case, the answer is 6.9%, which is correct, as we can see in the table from the budget review below (which is the same as the second source document in the markdown table above).

What do we learn?

There are some missing values, particularly in the series GDP (current ZAR[billions]) and Budget balance (% of GDP).

In addition, in 2019, the government expenditure figure is incorectly plucked from the PDF document as 29.40% of GDP, when in reality, this was 35.70% of GDP. This is a significant error, and it is not clear why this happened. It is possible that the PDF document was not correctly parsed, or that the OpenAI API made an error extracting this value from the text chunk.

In order to overcome these kinds of errors, experimenting with multiple pipelines and chunk sizes is probably the best way to go. This was just a demonstration, and there is certainly a possiblity to make the pipline more robust and less error prone.

Conclusion

Though we have used budget data from South Africa in this case, the same approach can be used for any kind of data. LangChain has document loaders for text, PDF, and HTML, and can be used to extract information from many kinds of document.

What is an instant hot water tap?

The instant hot water tap system consists of a small, pressurised reservoir that is heated to 110 degrees Celsius and a tap to dispense the water. The system can be installed below a kitchen sink to minimise the distance and potential heat loss between the tap and the reservoir. Because the reservoir is pressurised, the water is unable to boil, even at 110 degrees Celsius. When the tap is opened, the pressure is released, and the water boils instantly.

For the remainder of the blog, I will refer to this system as a Quooker, a dutch brand which produces these taps, about which I could find the most information on the internet.

How does it compare to a kettle?

Along with convenience, the Quooker also claims to be more energy efficient than a kettle by providing the exact amount of boiling water required for a cup of tea. This is in contrast to a kettle, which is often overfilled by the user, wasting energy and water.

My goal was thus to calculate for how long these energy savings would have to accrue before the Quooker became more cost-effective than a kettle. To do this, I ask GPT-4 to help me calculate the energy consumption of both the Quooker and a standard electric kettle.

Find out for yourself

You can adjust any of these assumptions in the inputs below to see how they affect the result. The entire article is dynamic, so adjust the inputs and read it again from the top to see how the calculations change and what the payoff period is in your case!²

import { aq, op } from '@uwdata/arquero'

viewof kettle_efficiency= Inputs.range(
  [50, 100], 
  {value: 70, step: 5, label: "Kettle efficiency:"}
)

viewof Quooker_efficiency= Inputs.range(
  [50, 100], 
  {value: 90, step: 5, label: "Quooker efficiency:"}
)

viewof cups_per_day= Inputs.range(
  [0, 40], 
  {value: 8, step: 1, label: "Cups of tea per day:"}
)

viewof kettle_overfill= Inputs.range(
  [0, 1000], 
  {value: 250, step: 50, label: "ml by which kettle is overfilled:"}
)

viewof Quooker_price= Inputs.range(
  [50, 2000], 
  {value: 1000, step: 50, label: "Price of Quooker (Euro):"}
)

viewof kettle_price= Inputs.range(
  [50, 2000], 
  {value: 50, step: 50, label: "Price of kettle (Euro):"}
)

viewof electricity_price= Inputs.range(
  [0, 1], 
  {value: .3, step: 0.05, label: "Price of electricity per kWh (Euro):"}
)

c = 4.184  // Specific heat capacity of water in J/g°C
T_change = 80  // Change in temperature for kettle (100°C - 20°C)
T_change_Quooker = 90  // Change in temperature for Quooker (110°C - 20°C)
water_density = 1  // Density of water in g/mL
mass_tea = 250 * water_density
mass_overfill = kettle_overfill * water_density
mass_kettle = mass_tea + mass_overfill
Q_kettle = (mass_kettle * c * T_change) / (kettle_efficiency / 100)  // Energy in joules
Q_kettle_kWh = Q_kettle / (3.6 * Math.pow(10, 6))  // Convert energy to kWh
total_kettle_energy = Q_kettle_kWh * cups_per_day // Total energy for kettle per day
Q_Quooker = (mass_tea * c * T_change_Quooker) / (Quooker_efficiency / 100)  // Energy in joules
Q_Quooker_kWh = Q_Quooker / (3.6 * Math.pow(10, 6))  // Convert energy to kWh
total_Quooker_energy = (Q_Quooker_kWh * cups_per_day) + 0.24  // Total energy for Quooker per day
kettle_cost_daily = total_kettle_energy * electricity_price
Quooker_cost_daily = total_Quooker_energy * electricity_price
kettle_cost_yearly = kettle_cost_daily * 365
Quooker_cost_yearly = Quooker_cost_daily * 365

kettle_cost_yearly_round = Math.round(kettle_cost_yearly * 100) / 100
Quooker_cost_yearly_round = Math.round(Quooker_cost_yearly * 100) / 100

Q_kettle_wasted = (mass_overfill * c * T_change) / (kettle_efficiency / 100)
Q_kettle_wasted_kWh = Q_kettle_wasted / (3.6 * Math.pow(10, 6))
total_kettle_energy_wasted = Q_kettle_wasted_kWh * cups_per_day
total_kettle_energy_wasted_round = Math.round(total_kettle_energy_wasted * 100) / 100
total_kettle_energy_wasted_cost_round = Math.round(total_kettle_energy_wasted * electricity_price * 365)
percentage_kettle_energy_wasted = Math.round(total_kettle_energy_wasted / total_kettle_energy * 100)

Q_Quooker_wasted = 0.24
total_Quooker_energy_wasted_cost_round = Math.round(Q_Quooker_wasted * electricity_price * 365)
percentage_Quooker_energy_wasted_round = Math.round(Q_Quooker_wasted / total_Quooker_energy * 100)

years = (Quooker_price - kettle_price) / (kettle_cost_yearly - Quooker_cost_yearly)
years_round = Math.round(years * 100) / 100

dt = aq.table({
  years: Array.from({length: 200}, (_, i) => i + 1),
  kettle: Array.from({length: 200}, (_, i) => {
    return (i + 1) * kettle_cost_yearly + kettle_price
  }),
  Quooker: Array.from({length: 200}, (_, i) => {
    return (i + 1) * Quooker_cost_yearly + Quooker_price
  })
})

// Prepare the data in a format suitable for Plot.line
data = dt.objects().flatMap(({years, kettle, Quooker}) => [
  {years, value: kettle, key: 'Kettle'},
  {years, value: Quooker, key: 'Quooker'}
]);

Plot.plot({
  y: {
    grid: true,
    label: 'Cumulative cost (Euro)'
  },
  x: {
    grid: true,
    label: 'Time (years)'
  },
  color: {
    domain: ["Kettle", "Quooker"], legend: true, type: "categorical", range: ["#1f77b4", "#ff7f0e"]
  },
  marks: [
    Plot.line(data, {x: 'years', y: 'value', stroke: 'key'})
  ]})

What else do we learn?

As the number of cups of tea increases, the Quooker becomes more cost-effective than the kettle, as the wasted energy is a fixed amount, while the wasted energy of the kettle increases linearly with the number of cups of tea, assuming the overfilling remains constant.

Reducing the amount by which we overfill the kettle decreases the energy loss and the cost of operating the kettle.

Appendix

I have assumed that the water in the kettle cools completely between each boil. This would require 2 hours between each boil, assuming an ambient temperature of 20 degrees Celsius and a cooling constant of 0.02. The cooling constant would change based on the specific kettle/environment. See the code below for the simulation.

import matplotlib.pyplot as plt
import numpy as np
from scipy.integrate import odeint

# Define the function to compute derivative
def model(T, t):
    T_ambient = 20  # Ambient temperature in degree Celsius
    k = 0.02  # Cooling constant. This would change based on the specific kettle/environment
    dTdt = -k * (T - T_ambient)
    return dTdt

T0 = 100  # Starting temperature of the water in degree Celsius

t = np.linspace(0, 120, num=500)  # Time points in minutes. Here we simulate for 2 hours

# Solve the ODE
T = odeint(model, T0, t)

# Plot the results
plt.plot(t, T)
plt.xlabel('Time (minutes)')
plt.ylabel('Temperature (°C)')
plt.title('Cooling of Boiled Water in a Kettle Over Time')
plt.grid(True)
plt.show()

How the Tool Works

The YouTube Caption Summaries tool is a streamlined tool that condenses the key points from a Youtube video transcript. As an input, you provide a Youtube URL, and the tool returns a text file with the summary. It makes use of the youtube-transcript-api to grab the transcript, and the OpenAI API to summarize it, leveraging the GPT-3.5 turbo model.

This tool isn’t limited to English content. If you’re dealing with a non-English video, you can specify the original language, and the tool will automatically grab the auto generated English transcript before summarizing it.

graph TB
  A[Input YouTube URL] --> B[YouTube Data API: Get Transcript]
  B --> C[OpenAI API: Summarize Transcript]
  C --> D[Save Summary to Text File]

Performance: Pros and Cons

The summary above does a good job of capturing the main points that Parrique makes in his speech.

One issue is the quality of the data being fed into the summariser. The captioning process on Youtube is optional - with many uploaders foregoing any captions. The auto generated translations are not perfect either. There is a fantastic 99 % invisible epdisode explaining the history of closed captioning on Youtube, as well as the various improvements that have been made over the years. It certainly isn’t perfect, however, even if the transcription isn’t 100% accurate, it’s still a great starting point for a summary. When the transript is processed by the OpenAI API, the model is looking for the key points and can overcome transcription errors that are not crucial to the main message.

Overall, I am pleased with the quality of the summaries, and in the future I might work on providing the title of the video and it’s description as additional context for the summarization model.

Journey of Development

The development of this tool is a testament to the power of generative AI and a bit of curiosity. I did it in an afternoon, chatting with my virtual programming partner; GPT-4 offering up Python code based on my instructions.

While I am not an expert python coder, I could have produced this tool from first principles a year ago (in the pre-generative pretrained model era). It could have required a couple days worth of work, familiarizing myself with the various APIs, putting together a python package structure.

Instead, I could glue the different parts together in a couple of hours, and spend the rest of the time tweaking the code to get the best results.

Implications

This is the kind of project I would have used in a job interview to showcase my skills. Back in the day, being able to develop a tool like this would have been a testament to one’s coding skills. However, given that I didn’t code this from first principles, could I still do the same today? Is the product less impressive, knowing that a significant part of the coding was handled by an AI model?

The answer, I believe, is that the landscape of software tooling is changing. What’s important is not that the code was generated with AI assistance, but that the tool was developed effectively and solves a real problem. Coding from first principles will always have its place, but leveraging AI tools, just like any other tools, demonstrates practical problem-solving skills and adaptability. It’s a recognition that we are moving into an era where working in partnership with AI will be an essential part of many jobs, not just in the tech industry.

Conclusion

Solving little problems with a bit of code and a really cheap API call is a lot of fun. I’m looking forward to seeing what other tools I can build with the OpenAI API.

Planning

Extracting occupational trajectories from the Swedish biographical source “Vem är Vem?” comprises a two-part process. The first part involves translating the Swedish text, which is full of abbreviations and contractions, into English using the natural language processing capabilities of ChatGPT. This step is crucial for making the data accessible and understandable to researchers who may not be familiar with Swedish (like myself!).

Once the text has been translated, the second part of the process demands structuring the data into year and occupation pairs. This involves identifying mentions of job titles and dates in the text and extracting them in a structured format.

ChatGPT as translator

I simply ask ChatGPT:

Can you convert the following text from Swedish into English? There are many abbreviations, for instance, “f.” means “född” or born, “Med. kand.” means “medicine kandidat”.

It is a biography about a man:

And I paste in the OCR output:

Abramson, Ernst Leopold, professor, Djursholm, f. i Grava, Värml. L, 12/3 /96 av godsäg. Hugo A. o. Anna Salén. Med. kand. Sthlm 18, med. lic. o. dr 27, doc. i fysiol. 27, labor. i fysiol. 30-38, prof. o. förest, v. statrs inst. f. folkhälsan fr. 38. Batalj :läk. i f alt-läk rkår. res. 27, lär. i fysiol, o. rörel-sefysiol. v. gymn. centr anst. 22-30, led. av arméförv :s sjukv rstyr. vetensk. råd 36, medl. av riksidr rförb. öv rstyr. 24-36, skolidr :förb. läk. 28, led. av direkt, öv. gymn. centr :inst. 33, ordf. där 36, led. o. sekr. i näringsråd. 36, ordf. i 36 års livsmedelslagstiftn :-sakk. Utg. ett 50-tal arb. särsk. rör. näringsfysiol., fysiol, mek., muskel-fysiol. RNO.

ChatGPT returns to me the translated text, free from abbreviations and contractions:

Abramson, Ernst Leopold, professor, Djursholm, born in Grava, Värmland, on March 12, 1896, to estate owner Hugo A. and Anna Salén. He obtained a medicine kandidat degree in Stockholm in 1918, followed by a medical licentiate and doctorate in 1927. He was a docent in physiology in 1927 and worked as a laboratory assistant in physiology from 1930 to 1938. From 1938, he was a professor and director at the State Institute for Public Health. He served as a battalion doctor and reserve officer from 1927 and was a teacher in physiology and movement physiology at the Gymnastics Central Institute from 1922 to 1930. He was a member of the Army Medical Corps and Reserve Force from 1924 to 1936, a member of the Scientific Council of the Army Medical Service from 1936, and a member of the School Management Association of Doctors from 1928. He was also a member of the Directorate of the Gymnastics Central Institute from 1933, its chairman from 1936, and a member and secretary of the Nutrition Council from 1936. In addition, he was the chairman of the 1936 Food Legislation Committee. He authored around 50 works, primarily related to nutrition physiology, physiology, mechanics, and muscle physiology. He was awarded the Royal Order of the North Star (RNO).

It surprises me that ChatGPT is able to extrapolate correctly from just “RNO” (an abbreviation of the Swedish Nordstjärneorden) in the final part of the description to:

He was awarded the Royal Order of the North Star (RNO).

Fantastic!

Next I ask ChatGPT to return to us the occupational information in a structured format.

ChatGPT as interpreter

Strucutred data is our goal. Again I use a simple instruction to ask:

Great, thank you, can you extract the occupations he held and the dates for which he held them, and return the output in an R tibble with a column for start year, a column for end year (if any), a column for the Swedish description of the occupation, and a column for the English translation.

It returns the following R code:

library(tidyverse)

occupations <- tibble(
  start_year = c(1918, 1927, 1927, 1930, 1938, 1927, 1922, 1924, 1936, 1928, 1933, 1936),
  end_year = c(NA, NA, NA, 1938, NA, 1936, 1930, 1936, NA, NA, NA, NA),
  swedish_occupation = c(
    "Med. kand.", "Med. lic. o. dr", "Docent i fysiol.", "Labor. i fysiol.",
    "Prof. o. förest, v. stats inst. f. folkhälsan", "Batalj :läk. i f alt-läk rkår.",
    "Lär. i fysiol, o. rörel-sefysiol. v. gymn. centr anst.", "Led. av arméförv :s sjukv rstyr.",
    "Medl. av riksidr rförb.", "Skolidr :förb. läk.", "Led. av direkt, öv. gymn. centr :inst.",
    "Ordf. i 36 års livsmedelslagstiftn :-sakk."
  ),
  english_occupation = c(
    "Medicine kandidat", "Medical licentiate and doctorate", "Docent in physiology",
    "Laboratory assistant in physiology", "Professor and director at the State Institute for Public Health",
    "Battalion doctor and reserve officer", "Teacher in physiology and movement physiology at the Gymnastics Central Institute",
    "Member of the Army Medical Corps and Reserve Force", "Member of the Scientific Council of the Army Medical Service",
    "Member of the School Management Association of Doctors", "Member of the Directorate of the Gymnastics Central Institute",
    "Chairman of the 1936 Food Legislation Committee."
  )
)

Conclusion

Thank you for reading along. I have demonstrated how we can use unstructured text data to reconstruct career trajectories for notable Swedes, translating and structuring the biographical text data through the ChatGPT web interface.

Next week I’ll share a post on how to take this process and scale it by leveraging OpenAI’s API.

I hope that it might prove useful in your own research!

Getting started

We load the tidyverse for data manipulation, the sf package fo working with maps and set the ggplot theme to a light one for more attractive plots.

library(tidyverse)
library(sf)
theme_set(theme_light())

Set the crs as the same

We’ll use EPSG 4326 or WGS84. We can use the st_transform() function, specifying the crs that we want for our new map object, which we name sweden_map_4326 to avoid confusion.

This is what the CRS now looks like for Sweden - the same as the one for Norway.

sweden_map_4326 <- st_transform(sweden_map, crs = 4326)

sweden_map_4326 %>% 
  st_crs()

Coordinate Reference System:
  User input: EPSG:4326 
  wkt:
GEOGCRS["WGS 84",
    DATUM["World Geodetic System 1984",
        ELLIPSOID["WGS 84",6378137,298.257223563,
            LENGTHUNIT["metre",1]]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["geodetic latitude (Lat)",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["geodetic longitude (Lon)",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]],
    USAGE[
        SCOPE["Horizontal component of 3D system."],
        AREA["World."],
        BBOX[-90,-180,90,180]],
    ID["EPSG",4326]]

Let’s plot them together

We can plot the together and choose some nice colours from the national flags for the fill aesthetic.

ggplot() +
  geom_sf(data = sweden_map_4326, fill = "#03558A") +
  geom_sf(data = norway_map, fill = "#B40C2E")

Write out the file for Sweden individually

We can save the shapefile for Sweden with the st_write() command if we want to keep the maps separate.

sweden_map_4326 %>% 
  st_write(here::here("posts", 
                      "2023-01-31-map-projections", 
                      "data", 
                      "sweden_map_4326.shp"), delete_dsn = TRUE)

Deleting source `C:/Users/User/Documents/Recon/interlude-one/posts/2023-01-31-map-projections/data/sweden_map_4326.shp' using driver `ESRI Shapefile'
Writing layer `sweden_map_4326' to data source 
  `C:/Users/User/Documents/Recon/interlude-one/posts/2023-01-31-map-projections/data/sweden_map_4326.shp' using driver `ESRI Shapefile'
Writing 1 features with 63 fields and geometry type Multi Polygon.

Conclusion

Here we learned how to make it possible to plot maps with two different projections together by making use of the st_transform() command in R. We learned how to save the two maps as separate features in a shapefile, as well as how to combine them together. We can import the shapefile into any other software for plotting.

Inspiration

🚀 I am inspired by the company Electricity Maps based in Copenhagen, who allow you to understand the electricity grid through data. They collect data from all around the world and compute the data in an hourly resolution, allowing you to gain granular insights accounting for the variance of the grid.💡 While we won’t use their data in this blog post, we will take a look at some great annual statistics compiled by the team at Our World in Data. This organization is dedicated to providing data-driven insights on global issues and their data on electricity generation per capita is a valuable resource for understanding how different countries are progressing in terms of energy production.📊

Finished product

Here is the finished product that we will create in the tutorial