This content originally appeared on DEV Community and was authored by Dave Shilander
How to make a CLI based movie scrapper using NodeJS
This guide will instruct you on how to make a CLI application that will ask the user for movie, and present info about that movie in return. To be most successful you should have at least a beginers understanding of Node and NPM. You should also have a basic understanding of using bash to navigate files. Although you are free to use a GUI application if you wish, only terminal examples will be provided.
Prerequsites
-
A MovieDB API Key
You will need a free acount with TheMovieDB to obtain and API key. You can sign up for one here
Once you have a key, you can consult the API Documentation -
Node JS
You will need to have Node JS installed. Instruction for various platforms can be found here
A text editor or IDE of your choice.
A terminal that is capable of running bash commands.
Part One (Setup Folder & File Structure, Install NPM Packages)
First, create the root folder for the project. This guide will call it movie-scrapper. Then navigate into the newly created folder.
mkdir movie-scrapper && cd movie-scrapper
Within the movie-scrapper directory run the following command.
npm init -y
This will create the package.json file. You need to make two modifications to this file.
Add "type": "module" to the top level, and "start": "node src/index.js" to the scripts section.
It should look like this. (You may have some slight differences depending on your pesoral npm settings, just focus on the changes needed above)
{
"name": "movie-scrapper",
"version": "1.0.0",
"description": "",
"main": "index.js",
"type": "module",
"scripts": {
"start": "node src/index.js",
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "",
"license": "ISC",
"dependencies": {
}
}
Run this next command in your terminal to install the npm packages axios dotenv and inquirer
npm i axios dotenv inquirer
You should now see a node-modules directory in your root folder.
.
├── node_modules
├── package-lock.json
├── package.json
You should also see the newly installed packages listed in your package.json dependcies
"dependencies": {
"axios": "^0.26.1",
"dotenv": "^16.0.0",
"inquirer": "^8.2.2"
}
Now you are going to create a .env file that will hold your API key for TheMovieDB. If you are using git or any other version control for this project, remember to not commit this file.
Replace PUT_YOUR_API_KEY_HERE with your actual API key. Keep the backslashes and quotation marks.
echo TMDB_API_KEY=\"PUT_YOUR_API_KEY_HERE\" >> .env
You can verify the file was created using the cat command
cat .env
You will see th following output with your API key in place of the Xs
TMDB_API_KEY="XXXXXXXXXX"
Create the src directory that will contain the logic for the program, and navigate into it.
mkdir src && cd src
Create the the following files in the src directory:
Movie.js
config.js
getByID.js
index.js
prompt.js
searchMovie.js
You can do this in one command in your terminal
touch {Movie,config,getByID,index,prompt,searchMovie}.js
Your project should now have the following file stucture
.
├── .env
├── node_modules
├── package-lock.json
├── package.json
└── src
├── Movie.js
├── config.js
├── getByID.js
├── index.js
├── prompt.js
└── searchMovie.js
Part Two (Javascript Files)
Copy/Paste the following code to each correspodning file
src/config.js
import dotenv from "dotenv";
dotenv.config();
export default{
tmbdkey: process.env.TMDB_API_KEY,
};
src/getByID.js
import config from "./config.js";
import axios from 'axios'
const getByID = async (id) => {
const options={
params: {
api_key: config.tmbdkey,
language: "en-US",
append_to_response: `credits`
}
}
let response = await axios.get(`https://api.themoviedb.org/3/movie/${id}`, options)
return response.data
}
export default getByID;
src/index.js
import inquirer from 'inquirer';
import Movie from './Movie.js'
import moviePrompts from './prompt.js'
const movieResponse = await inquirer.prompt(moviePrompts)
const selectedMovie = await new Movie(movieResponse.movieID);
selectedMovie.summary()
src/Movie.js
import getByID from './getByID.js'
class Movie {
constructor(id) {
return (async () => {
this.details = await getByID(id)
return this
})()
}
get title() {
return this.details.title
}
get tagline() {
return this.details.tagline
}
get overview() {
return this.details.overview
}
get directors() {
const directors = this.details.credits.crew.filter(
(obj) => obj.job === 'Director',
)
return directors.map((director) => director.name)
}
get writers() {
const writers = this.details.credits.crew.filter(
(obj) => obj.job === 'Screenplay',
)
return writers.map((writer) => writer.name)
}
get cast() {
const cast = this.details.credits.cast.slice(0, 5)
return cast.map((castMember) => ({
name: castMember.name,
role: castMember.character,
}))
}
summary() {
const summary =
`
${this.title} - ${this.tagline}
Directed By: ${this.directors} Written By: ${this.writers}
${this.overview}
Starring:
${this.cast.map(castMember => ` ${castMember.name} as ${castMember.role}`)}
`
console.log(summary)
}
}
export default Movie
src/prompt.js
import searchMovie from './searchMovie.js'
const moviePrompts = [
{
name: "name",
type: "input",
message: "Enter a movie to search:"
},
{
name: "movieID",
type: "list",
message: "Select a movie:",
choices: (answers) => searchMovie(answers.name)
}
]
export default moviePrompts
src/searchMovie.js
import config from "./config.js";
import axios from 'axios'
const searchMovie = async (movie) => {
const options={
params: {
api_key: config.tmbdkey,
language: "en-US",
query: movie,
}
}
const simpleList = movieObj => ({name: `${movieObj.title} (${movieObj.release_date.slice(0,4)})`, value: movieObj.id})
const res = await axios.get('https://api.themoviedb.org/3/search/movie', options);
const list = res.data.results.map(simpleList).slice(0,5)
return list;
}
export default searchMovie;
Part 3 (Execution)
At the root of your project run
npm start
You will be preseted with the following.
> movie-scrapper@1.0.0 start
> node src/index.js
? Enter a movie to search:
Follow the prompts as given by the application.
This content originally appeared on DEV Community and was authored by Dave Shilander
Dave Shilander | Sciencx (2022-05-02T02:03:18+00:00) How to make a CLI based movie scrapper using NodeJS. Retrieved from https://www.scien.cx/2022/05/02/how-to-make-a-cli-based-movie-scrapper-using-nodejs/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.