Tagging algorithm problems from statement and code

~0.68macro-F1 on the frozen test set

~0.65zero-shot LLM, for comparison

~5,000public problems

[ Architecture ]

How it works

Run a scenario to follow a request through the system. Select a component to read its role and its flows.

question

A graph problem with a Python solution

A statement about connected components, with a depth-first search solution.

solution.py

ConsoleSimulated run · illustrative data

Press Classify a problem to watch the request go through the system.

01ProblemStatement and solution received
02TF-IDF + linearText and code scored by the linear model
03Fusion & thresholdsFirst branch scores held
04ModernBERTTransformer scores the same tags
05Fusion & thresholdsScores blended · each tag checked against its threshold
06TagsPredicted tags - graphs · dfs
✓The problem gets its predicted tags. On the frozen test set this pipeline reaches a macro-F1 of 0.683.

Problem

Predict eight algorithm tags (multi-label) from the statement of a programming problem and a Python solution. The evaluation protocol matters as much as the model.

Data

About 5,000 public problems from a competitive-programming site, with statements, Python solutions and tags. The labels are incomplete, which caps the achievable score.

Approach

Protocol first. The test set is frozen before any model is built, and models are chosen by cross-validation with near-duplicates grouped to avoid leakage.
Two branches, blended. A linear model on TF-IDF features and a fine-tuned ModernBERT, with a threshold tuned per tag.
A baseline. A zero-shot LLM on the same task.

Evaluation

The blend reaches about 0.68 macro-F1 on the frozen test set, slightly above the zero-shot LLM (about 0.65). Incomplete labels cap the score.

Result

A small, reproducible pipeline, scored once on held-out data, with no model call per prediction.

My role

A personal project, built on my own, from the protocol to the LLM baseline.