1. DeepPavlov for developers: # 1 NLP tools and create chat bots
2. DeepPavlov for developers: # 2 setup and deployment
Hello! In the first article in our series, we learned what DeepPavlov is, what library models are ready for use without prior training, and how to run REST servers with them. Before starting the training of models, we will talk about the various possibilities of deploying DeepPavlov models and some features of the library settings.
We agree that all the library startup scripts are executed in the Python environment with the DeepPavlov library installed (for installation, see the first article, for virtualenv , read here ). The examples in this article do not require knowledge of Python syntax.
Modes of interaction with NLP models DeepPavlov
DeepPavlov currently supports 4 ways to interact with NLP models (both pre-trained and user-created):
- REST server ( riseapi mode) is the main tool for integrating models, it was described in detail in a previous article (documentation) .
- TCP or UNIX socket server ( risesocket mode) - if low-level integration (documentation) is required.
- Bot in Telegram ( telegram mode) - demo mode, allows you to interact with the model through Telegram (documentation) .
- Command line ( interact mode) - demo and debug mode, allows you to interact with the model through the command line. The model in the interaction mode via the command line is initialized with the following command:
python -m deeppavlov interact
The <config_path> parameter (required in all four modes) can take either the full path to the model config file or the name of the config file without an extension. In the latter case, the model config must be registered in the library.
Configs of all models supplied with DeepPavlov are registered in the library. The list of supplied models can be found in the MODELS section of the DeepPavlov documentation , their configs can be found here .
GPU usage
In any of the above modes, NLP models are initialized, which are based on neural networks. This makes them quite demanding on computing resources. You can improve the performance of models by using the GPU. To do this, you will need an nVidia graphics card with sufficient video memory (depending on the model you are running) and a supported version of the CUDA framework. You can find all the necessary information on launching DeepPavlov models on the GPU here .
Library settings files
All library settings are contained in three files:
- server_config.json - REST and socket server settings, as well as a Telegram connector
- dialog_logger_config.json - settings for logging requests to models
- log_config.json - library logging settings
By default, the configuration files are located in <deep_pavlov_root> / utils / settings , where <deep_pavlov_root> is the installation directory of DeepPavlov (usually this is lib / python <XX> / site-packages / deeppavlov in virtual environment). Using command
python -m deeppavlov.settings
You can find out the exact path to the directory with the settings files. You can also set the path to a directory convenient for you by specifying it in the environment variable DP_SETTINGS_PATH . After the first run of the above command (a server with any trained DeepPavlov model), files from the default directory will be copied to the directory from DP_SETTINGS_PATH . Team
python -m deeppavlov.settings -d
resets settings by copying settings files from the default directory on top of files to DP_SETTINGS_PATH .
From the settings of DeepPavlov you should pay attention to:
- server_config.json , parameter model_args_names:
From the last article we remember:
- arguments to the REST API DeepPavlov named;
- any model in DeepPavlov is identified by the name of its config.
So, the default argument names for each model are taken from its config.
We will analyze the structure of model configs in detail in the following articles, now we only note that the argument names in the REST API can be redefined as follows:
model_args_names: [“arg_1_name”, ..., “arg_n_name”]
The sequence of argument names corresponds to the sequence in which the arguments are defined in the model config, the empty string as the value of the model_args_names parameter corresponds to the default names. - log_config.json:
Please note that for logging uvicorn a logger is used, which is configured separately. You can read about the configuration structure of the logging Python module here .
Running pre-trained models in Docker
Any pre-trained DeepPavlov model can be launched in the Docker container in the REST service mode. Detailed instructions are in our repositories on DockerHub: here for the CPU, here for the GPU. API models in containers are fully consistent with the description from the previous article.
Deeppavlov cloud
To make it easier to work with pre-trained NLP models from DeepPavlov, we started providing them in SaaS mode. To use the models, you need to register in our service and get a token in the Tokens section of your personal account. The API documentation is in the Info section. Under one token, you can send up to 1000 requests to the model.
Currently, the service is launched in the Alpha version and its use is provided free of charge. Further, the set and format for the provision of models will be expanded in accordance with user requests. The request form can be found at the bottom of the Demo page.
The following models are now available in DeepPavlov Cloud:
- Named Entity Recognition (multilingual) - recognition of named entities;
- Sentiment (RU) - classification of tonality of the text;
- SQuAD (multilingual) - the answer to a question to the text as a fragment of this text.
Conclusion
In this article, we got acquainted with the features of configuration and deployment of DeepPavlov models, learned about Docker DP images and the possibility of free access to DP models as SaaS.
In the next article, we will train a simple DeepPavlov model on our dataset. And do not forget that DeepPavlov has a forum - ask your questions regarding the library and models. Thanks for attention!