nltk - Download data models while installing my python library - Stack Overflow

admin管理员组

文章数量:1289582

Sometimes, a Python library depends on additional data, such as ML models. This could be a model from transformers, spacy, nltkand so on. Typically there is a command to download such a model:

python -m nltk.downloader stopwords

How can I have this done automatically when my library is installed, when I use modern python packaging (pyproject.toml) and tools (uv)? The reason is that my library is used in another application and that application shouldn't be concerned what models I use under the hood. But I also don't want to download them at run time. AFAIK, there is no post install step anymore.

I could provide a post install command in my package that the user would have to manually execute once. Is there any convention for this?

At least for spacy, the models are distributed as wheels. However, they can't be used as dependencies by uv since they are lacking versions.

Sometimes, a Python library depends on additional data, such as ML models. This could be a model from transformers, spacy, nltkand so on. Typically there is a command to download such a model:

python -m nltk.downloader stopwords

How can I have this done automatically when my library is installed, when I use modern python packaging (pyproject.toml) and tools (uv)? The reason is that my library is used in another application and that application shouldn't be concerned what models I use under the hood. But I also don't want to download them at run time. AFAIK, there is no post install step anymore.

I could provide a post install command in my package that the user would have to manually execute once. Is there any convention for this?

At least for spacy, the models are distributed as wheels. However, they can't be used as dependencies by uv since they are lacking versions.

• python
• nltk
• spacy
• pyproject.toml
• uv

Share Improve this question Follow asked Feb 20 at 7:21 jdmjdm 10.1k1313 gold badges6767 silver badges116116 bronze badges

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 1

When using modern packaging you can not execute arbitrary code during installation. I suggest you have the package check the availability of the necessary dependencies at entrypoint/import time and inform the user. Based on your error message the user can then call a different entrypoint/function that will install the necessary dependencies. This installation should only run if the missing dependencies are not already found.

本文标签： nltkDownload data models while installing my python libraryStack Overflow