<h2>Overview</h2>
<p>FLASK (Fine-grained Language Model Evaluation Based on Alignment Skill Sets) is a task-agnostic evaluation protocol for language models, focusing on fine-grained instance-wise skill sets as metrics. This repository serves as the official platform for the FLASK project, providing model-based and human-based evaluation guidelines.</p>
<h2>Features</h2>
<ul>
<li>Task-agnostic evaluation protocol for language models</li>
<li>Implementation of fine-grained instance-wise skill sets as metrics</li>
<li>Support for OpenAI API key integration for GPT-4 evaluation</li>
<li>Model inference and evaluation functionalities</li>
<li>Aggregation and analysis capabilities based on skills, domains, and difficulty levels</li>
<li>Metadata annotation implementation for domain and skillset categorization</li>
</ul>


Kaistai

FLASK