| LangKit is an open source text metrics toolkit for monitoring language models. It provides a set of methods for extracting relevant signals from input and/or output text, which are compatible with the open source datalogging library whylogs. Currently supported metrics include: Text Quality Readability Score Complexity and Grade Score Text Relevance Similarity Score Between Prompts/Responses Similarity Score to User-Defined Topics Security and Privacy patterns – match user-defined regex patterns Group String Count Jailbreak – Similarity Score to Known Jailbreak Attempts Hint Injection – Vs Known Hints… |
#large #language #models #safe #reliable #LangKit