Posted on May 3, 2023

How to install Python packages without a requirements.txt file with pipreqs

#python #programming #productivity #tutorial

Why bother

Say you get a project that doesn't have a requirements.txt file in it and that project has 20+ imports, meaning you need to install 20+ modules manually. Sound not that interesting.

That's when pipreqs comes into play as a "life saver". This tool will scan all scripts/folders in the current working directory (or where you want it to look by providing a path) and installs all the found packages.

Example usage

Solution

create virtual env
activate it
install pipreqs
tell pipreqs to look for files in the current folder "./" and use --encoding utf-8
- wait until requirements.txt is created
install script dependencies from created requirements.txt

Which results in this command:

# windows python -m venv env && \ source env/Scripts/activate && \ pip install pipreqs && \ pipreqs --encoding utf-8 "./" && \ pip install -r requirements.txt && \ pip freeze > requirements.txt

# linux python -m venv env && \ source env/source/activate && \ pip install pipreqs && \ pipreqs --encoding utf-8 "./" && \ pip install -r requirements.txt && \ pip freeze > requirements.txt

Low amount of imports example

Let's say you have a script like this:

import requests response = requests.get('https://serpapi.com/playground') print(response.html)

Big amount of imports example

The point of it is to show how all the modules install automatically without having an initial requirements.txt file.

Here will have a bigger amount of imports:

import pandas as pd import numpy as np import matplotlib.pyplot as plt import seaborn as sns import plotly.express as px import scipy.stats as stats import statsmodels.api as sm import sklearn import yellowbrick import wordcloud import nltk import spacy import transformers import streamlit as st # Load and clean data data = pd.read_csv('data.csv') data.dropna(inplace=True) # Descriptive statistics print('Data Summary') print(data.describe()) # Data visualization sns.histplot(data['age'], kde=False, bins=10) plt.title('Age Distribution') plt.show() px.scatter(data, x='income', y='age', color='gender', title='Income vs. Age') # Correlation analysis corr_matrix = data.corr() sns.heatmap(corr_matrix, annot=True, cmap='coolwarm') plt.title('Correlation Matrix') plt.show() # Statistical analysis stat, p = stats.ttest_ind(data[data['gender']=='M']['income'], data[data['gender']=='F']['income']) print(f'T-test: statistic={stat}, pvalue={p}') # Machine learning X = data[['age', 'income']] y = data['gender'] model = sklearn.linear_model.LogisticRegression() model.fit(X, y) visualizer = yellowbrick.classifier.classification_report(model, X, y) visualizer.show() # Text analysis text = 'This is a sample text for text analysis' tokens = nltk.word_tokenize(text) print(f'Tokenized text: {tokens}') nlp = spacy.load('en_core_web_sm') doc = nlp(text) for token in doc: print(token.text, token.pos_) model = transformers.pipeline('sentiment-analysis') result = model(text)[0] print(f'Sentiment analysis: {result["label"]}, score={result["score"]}') # Streamlit app st.title('Data Analysis App') st.write('Data Summary') st.write(data.describe())

Generated requirements.txt afterward:

altair==4.2.2 attrs==23.1.0 blinker==1.6.2 blis==0.7.9 cachetools==5.3.0 catalogue==2.0.8 certifi==2022.12.7 charset-normalizer==3.1.0 click==8.1.3 colorama==0.4.6 confection==0.0.4 contourpy==1.0.7 cssselect==1.2.0 cycler==0.11.0 cymem==2.0.7 decorator==5.1.1 docopt==0.6.2 entrypoints==0.4 filelock==3.12.0 fonttools==4.39.3 fsspec==2023.4.0 gitdb==4.0.10 GitPython==3.1.31 huggingface-hub==0.14.1 idna==3.4 importlib-metadata==6.6.0 Jinja2==3.1.2 jmespath==1.0.1 joblib==1.2.0 jsonschema==4.17.3 kiwisolver==1.4.4 langcodes==3.3.0 lxml==4.9.2 markdown-it-py==2.2.0 MarkupSafe==2.1.2 matplotlib==3.7.1 mdurl==0.1.2 murmurhash==1.0.9 nltk==3.8.1 numpy==1.24.3 packaging==23.1 pandas==2.0.1 parsel==1.8.1 pathy==0.10.1 patsy==0.5.3 Pillow==9.5.0 pipreqs==0.4.13 plotly==5.14.1 preshed==3.0.8 protobuf==3.20.3 pyarrow==12.0.0 pydantic==1.10.7 pydeck==0.8.1b0 Pygments==2.15.1 Pympler==1.0.1 pyparsing==3.0.9 pyrsistent==0.19.3 python-dateutil==2.8.2 pytz==2023.3 pytz-deprecation-shim==0.1.0.post0 PyYAML==6.0 regex==2023.5.4 requests==2.29.0 rich==13.3.5 scikit-learn==1.2.2 scipy==1.10.1 seaborn==0.12.2 six==1.16.0 smart-open==6.3.0 smmap==5.0.0 spacy==3.5.2 spacy-legacy==3.0.12 spacy-loggers==1.0.4 srsly==2.4.6 statsmodels==0.13.5 streamlit==1.22.0 tenacity==8.2.2 thinc==8.1.10 threadpoolctl==3.1.0 tokenizers==0.13.3 toml==0.10.2 toolz==0.12.0 tornado==6.3.1 tqdm==4.65.0 transformers==4.28.1 typer==0.7.0 typing_extensions==4.5.0 tzdata==2023.3 tzlocal==4.3 urllib3==1.26.15 validators==0.20.0 w3lib==2.1.1 wasabi==1.1.1 watchdog==3.0.0 wordcloud==1.9.1.1 yarg==0.1.9 yellowbrick==1.5 zipp==3.15.0