Install third party python libraries (e.g. PDFMiner)

I would like to use third party python libraries in a server whitelist script.
The thing is: by running pip pip install pdfminer I am unable to use this library in a py file:

Failed to get method for command
xxx.xxx.doctype.yyy.pdf_data.pdfmetadata with No module named 'pdfminer'

Is it even possible to import third party libraries using installation manager that fixes all prerequisites?

1 Like

You’ll have to activate the virtual environment first:

# change to bench directory
cd /home/frappe/frappe-bench
# activate virtual environment
source env/bin/activate
# install python package in virtual environment
pip install pdfminer
12 Likes

Thanks!
Haven’t had a chance to handle virtual environments “hands on” by myself by now.

And after the installation we don’t need to activate environment again?
How about in production?

  • Frappe/ERPNext automatically activates the 'env' virtual environment on startup. You don’t need to activate manually yourself.

  • It works the same in production.

  • If you don’t want to manually 'pip install ...' in every environment? Then you would add your Python package requirements to your App’s “requirements.txt” file. (packages named in “requirements.txt” are automatically installed when you install the App)

As an example, here’s a link to the ERPNext App’s 'requirements.txt'.

4 Likes

thank you @brian_pond

If you’re using a relatively recent version of bench you can also do bench pip install PDFMiner

Hi @brian_pond , I am using hijri_converter package in an doc_event script that I mentioned in my hooks.py file.

I was able to solve the package error locally by doing bench pip install hijri_converter.

My issue is that in my production environment only migrations are applied based on the things in the hooks.py file that I push to prod using the export-fixtures command.

To summarize my issue, app installation don’t happen in my prod. environment every time, only migration is carried out.

Your third party package, “hijri_converter”, must be specified as a requirement for your App. You do this by adding its name to either "requirements.txt" or "pyproject.toml" (depending on Frappe framework version, or how you’ve built your App).

By adding “hijri_converter” to that file, it will automatically install itself when you install your App in Production.

I don’t have any advice for you about migrations, hooks.py, or fixtures. I don’t normally use those in my work. Another forum user could offer you better advice.

3 Likes

Thank you @brian_pond

Hello, how to add in pyproject.toml

@Abhiraj_Tulsyan - Here is a link to documentation about adding requirements to pyproject.toml

https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#dependencies-and-requirements