# How To Consume an External API with Express.js By Bobby Iliev, 2024-11-18. https://bobbyiliev.com/post/how-to-consume-an-external-api-with-expressjs/ # Introduction In this guide, we'll learn how to use **Node.js**, **Express**, and **Neon Postgres** to build a simple application that consumes the **QuizAPI** to fetch tech-related questions and store them in a PostgreSQL database. By the end of this guide, you'll be able to: 1. Use the **axios** library to make HTTP requests in Node.js. 2. Set up an **Express** server to handle routing. 3. Connect to a **Neon Postgres** database to store and retrieve data. 4. Deploy your application to a **DigitalOcean Droplet**. We'll even include storing the correct answers, so you can build on this project to create a full-fledged quiz application. # Prerequisites Before starting, make sure you have: - **Node.js** and **npm** installed on your system. You can download them [here](https://nodejs.org). - A **Neon Postgres** database. Sign up for a free account [here](https://neon.tech). - A **QuizAPI** account with an API key. You can sign up [here](https://quizapi.io). - Optionally, a **DigitalOcean Droplet** for deployment. Use my affiliate link to get **$200 in free credit**: [DigitalOcean Free Credit](https://m.do.co/c/2a9bba940f39). If you're new to setting up a Node.js application on a server, check out this guide: [How to Deploy a Node.js App on DigitalOcean](https://www.digitalocean.com/community/tutorials/how-to-set-up-a-node-js-application-for-production-on-ubuntu-22-04). --- # Project Setup Let's get started by creating a new Node.js project and setting up the necessary dependencies. ## Step 1: Initialize the Project Open your terminal and create a new directory for your project: ```bash mkdir node-quiz-api cd node-quiz-api npm init -y ``` The `-y` flag automatically fills in the default settings in your `package.json` file. ## Step 2: Install Dependencies We need a few libraries to make our project work: ```bash npm install express axios pg dotenv ``` Here's a quick overview of the packages: - **express**: A minimalist web framework for Node.js. - **axios**: A popular HTTP client for making API requests. - **pg**: PostgreSQL client to connect to our Neon database. - **dotenv**: Loads environment variables from a `.env` file. # Step 3: Setting Up the Database We'll use **Neon Postgres** to store our quiz data. Make sure you’ve created your database and have the connection details ready. ### Create a `.env` File In the root of your project, create a file named `.env`: ``` DATABASE_URL=postgres://:@:/ QUIZ_API_KEY=YOUR_QUIZ_API_KEY ``` Replace ``, ``, ``, ``, and `` with your actual Neon credentials. Don't forget to replace `YOUR_QUIZ_API_KEY` with your QuizAPI key. ### What is the `.env` file? The `.env` file allows you to securely store sensitive information like your database credentials and API keys. In production, a [secrets manager](https://devops-daily.com/posts/secrets-management-guide) is a safer home for them. It prevents hardcoding these details in your code, reducing security risks. --- # Step 4: Creating the Database Table To store questions and their answers, we'll create a table in our Neon database. ### Connect to Your Database You can connect to your Neon Postgres database using a tool like `psql`, DBeaver, or any other Postgres client. ### Create the `questions` Table Run the following SQL command: ```sql CREATE TABLE questions ( id SERIAL PRIMARY KEY, question TEXT NOT NULL, answer_a TEXT, answer_b TEXT, answer_c TEXT, answer_d TEXT, correct_a BOOLEAN, correct_b BOOLEAN, correct_c BOOLEAN, correct_d BOOLEAN, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ); ``` ### Explanation of the Table Structure: - `id`: A unique identifier for each question. - `question`: The quiz question text. - `answer_a`, `answer_b`, `answer_c`, `answer_d`: Possible answer options. - `correct_a`, `correct_b`, `correct_c`, `correct_d`: Boolean values indicating whether each answer is correct. - `created_at`: A timestamp for when the question was added. --- # Step 5: Setting Up the Project Files Let's create the file structure for our application: ``` node-quiz-api ├── .env ├── app.js ├── db.js └── package.json ``` ### `db.js` This file handles our connection to the Neon Postgres database: ```js const { Pool } = require('pg'); require('dotenv').config(); const pool = new Pool({ connectionString: process.env.DATABASE_URL, ssl: { rejectUnauthorized: false }, }); module.exports = { query: (text, params) => pool.query(text, params), }; ``` ### Explanation: - We’re using the `pg` package to connect to our database. - The `Pool` class manages multiple connections efficiently. - We use `dotenv` to load environment variables from our `.env` file. --- ### `app.js` This file sets up our Express server, handles requests to the QuizAPI, and stores the results in our database. ```js const express = require('express'); const axios = require('axios'); const db = require('./db'); require('dotenv').config(); const app = express(); const PORT = process.env.PORT || 3000; // Fetch and store questions from QuizAPI app.get('/fetch', async (req, res) => { try { const response = await axios.get('https://quizapi.io/api/v1/questions', { params: { apiKey: process.env.QUIZ_API_KEY, limit: 10, }, }); const quizzes = response.data; for (const quiz of quizzes) { const { question, answers, correct_answers } = quiz; await db.query( `INSERT INTO questions (question, answer_a, answer_b, answer_c, answer_d, correct_a, correct_b, correct_c, correct_d) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)`, [ question, answers.answer_a, answers.answer_b, answers.answer_c, answers.answer_d, correct_answers.answer_a_correct === 'true', correct_answers.answer_b_correct === 'true', correct_answers.answer_c_correct === 'true', correct_answers.answer_d_correct === 'true', ] ); } res.send('Questions fetched and stored successfully!'); } catch (error) { console.error('Error:', error.message); res.status(500).send('Failed to fetch questions.'); } }); // Retrieve all stored questions app.get('/questions', async (req, res) => { try { const result = await db.query('SELECT * FROM questions ORDER BY id DESC'); res.json(result.rows); } catch (error) { console.error('Error:', error.message); res.status(500).send('Failed to fetch questions.'); } }); app.listen(PORT, () => { console.log(`Server running on http://localhost:${PORT}`); }); ``` ### Explanation: 1. **Fetching Data**: We use `axios` to make a request to the QuizAPI. 2. **Storing Data**: We loop through the questions and store them in our Postgres database. 3. **Retrieving Data**: The `/questions` endpoint returns all stored questions. --- # Step 6: Running the Application To run your application: ```bash node app.js ``` Visit: - `http://localhost:3000/fetch` to fetch and store questions. - `http://localhost:3000/questions` to view the stored questions. --- # Step 7: Deploying Your Application to DigitalOcean Now that your application is working locally, let's deploy it to a server so it's accessible to others. We'll use a **DigitalOcean Droplet** for this purpose. If you're new to deploying Node.js applications, check out this comprehensive guide: **[How To Set Up a Node.js Application for Production on Ubuntu 16.04](https://www.digitalocean.com/community/tutorials/how-to-set-up-a-node-js-application-for-production-on-ubuntu-16-04)**. ### Quick Deployment Steps: 1. **Create a Droplet**: Use my [DigitalOcean link](https://m.do.co/c/2a9bba940f39) to get **$200 in free credit** and spin up an Ubuntu Droplet. 2. **SSH into Your Droplet**: ```bash ssh root@your_droplet_ip ``` 3. **Install Node.js**: Follow the steps in the linked guide to install Node.js, Nginx, and set up a firewall. 4. **Clone Your Project**: ```bash git clone https://github.com/your-username/node-quiz-api.git cd node-quiz-api ``` 5. **Install Dependencies**: ```bash npm install ``` 6. **Set Up Environment Variables**: Create a `.env` file on the server and copy your local `.env` content into it. 7. **Start Your Application**: ```bash node app.js ``` 8. **Use PM2 for Process Management**: To keep your Node.js app running in the background, install `pm2`: ```bash npm install -g pm2 pm2 start app.js ``` 9. **Set Up Nginx**: Use Nginx as a reverse proxy to route incoming requests to your Node.js app. Follow the configuration steps in the [DigitalOcean guide](https://www.digitalocean.com/community/tutorials/how-to-set-up-a-node-js-application-for-production-on-ubuntu-16-04). 10. **Access Your App**: Once everything is set up, you can access your application via your server's IP address. Use my [DigitalOcean link](https://m.do.co/c/2a9bba940f39) to get $200 in free credit! --- # Conclusion Here is how our Node.js application consumes the QuizAPI and stores the data in the Neon Postgres database: ``` +------------------------------------+ | QuizAPI Service | | https://quizapi.io/api/v1/questions| +------------------------------------+ ^ | (HTTP GET Request) | v +------------------------------------------------------+ | Node.js Application | |------------------------------------------------------| | 1. Express server receives request at '/fetch' route | | 2. Uses Axios to fetch questions from QuizAPI | | 3. Processes the response data | | 4. Stores questions and answers in the database | +------------------------------------------------------+ | | (Database Query) | v +----------------------------------------+ | Neon Postgres Database | |----------------------------------------| | Stores: | | - Question text | | - Answer options (A, B, C, D) | | - Correct answer flags (true/false) | +----------------------------------------+ ^ | (Database Query) | v +----------------------------------------------------------+ | Node.js Application retrieves questions via '/questions'| +----------------------------------------------------------+ | | v +-------------------------------+ | Client Browser | | (Displays stored data) | +-------------------------------+ ``` With that we've built a Node.js application that fetches data from an external API and stores it in a Neon Postgres database. You can now extend this project by adding a frontend, implementing authentication, or even creating a full quiz game.