Nowadays, there are pre-trained AI models for almost any use case. There are of course several takeaways that you should keep in mind whilst reusing these publicly available models.
When using pre-trained models, it’s important to ensure that the pre-training has been done on appropriate data, that matches your use case. For instance, check if it was the right language and that the characteristics of the texts are similar to the texts you want to process.
Your production data needs to follow the same preprocessing steps as were used when the model was trained. Not doing so will severely impact the model performance or not work at all. For instance, if you’re working with text data, look up what preprocessing has been conducted (lemmatisation, stop word removal, tokenisation, etc.). When using Huggingface models, use the tokenizer class that comes with your model.
Interacting with AI models can be challenging. Ensure users understand what is happening and how they should interpret the results of the model. Every AI model has some level of uncertainty on the accuracy of the output, be sure to make end-users aware of that.
If you need a custom-tailored solution based on text, you can still leverage the power of pre-trained AI models by using them for feature extraction. Use the intermediate representations produced by a model and combine these with your own model for the best of both worlds.
Always monitor the outputs of your model continuously and check if the model is used appropriately. Keeping track of outliers and corrected outputs is a great way to ensure possible fine-tuning of the models later on.
Your message is in good hands. We strive to get back at you within one working day.
