# eval: data_pipeline

*Community 5 | 4 files | cohesion 0.60*

## Definition

This community groups 4 file(s) rooted at `eval` with dominant language py (cohesion 0.60). Central symbols: `SandboxConfig`, `TeacherClient`, `__init__`, `_blocked_dunder_access`, `_build_worker_src`, `_max_depth`, `_names_imported`, `chat`. Core file: `topogpt3/data_pipeline.py` (16 symbols). Documented purpose: Sandbox for executing model-generated code during HumanEval evaluation.  This module replaces the bare `exec()` call in `eval/harness.py:run_one_test` with a de.

## Files

| File | Language | Layer | Symbols | Doc |
|------|----------|-------|---------|-----|
| `eval/sandbox.py` | py | utility | 9 | yes |
| `eval/sandbox_smoke.py` | py | utility | 1 | yes |
| `tests/test_data_pipeline.py` | py | testing | 6 | yes |
| `topogpt3/data_pipeline.py` | py | data_access | 16 | yes |

## Key Symbols

- `SandboxConfig` (class, `eval/sandbox.py:53`) `class SandboxConfig` - One knob per defence layer. Defaults match HumanEval-style eval.
- `_names_imported` (function, `eval/sandbox.py:100`) `def _names_imported(tree)` - Return the set of top-level names brought into scope by imports.
- `_blocked_dunder_access` (function, `eval/sandbox.py:114`) `def _blocked_dunder_access(tree, blocked)` - Find Attribute nodes whose attr is in `blocked`. Returns attr names found.
- `_max_depth` (function, `eval/sandbox.py:123`) `def _max_depth(tree)` - Compute max nesting depth of the AST. Catches obfuscated huge trees.
- `d` (function, `eval/sandbox.py:125`) `def d(node, cur)`
- `check_safety` (function, `eval/sandbox.py:133`) `def check_safety(source, cfg)` - Return (ok, reason). `reason` is "" when ok, else a human-readable
- `_build_worker_src` (function, `eval/sandbox.py:254`) `def _build_worker_src(allowed_builtin_names, program_src, blocked_modules)`
- `safe_exec` (function, `eval/sandbox.py:270`) `def safe_exec(program_src, cfg, extra_globals)` - Execute `program_src` in a sandboxed child process. Returns the same
- `describe_policy` (function, `eval/sandbox.py:373`) `def describe_policy(cfg)`
- `main` (function, `eval/sandbox_smoke.py:15`) `def main()`
- `test_normalize_strips_control_and_collapses_space` (function, `tests/test_data_pipeline.py:15`) `def test_normalize_strips_control_and_collapses_space()`
- `test_dedup_exact_and_fuzzy` (function, `tests/test_data_pipeline.py:23`) `def test_dedup_exact_and_fuzzy()`
- `test_quality_filter_bounds` (function, `tests/test_data_pipeline.py:35`) `def test_quality_filter_bounds()`
- `test_verify_python_ast_and_sandbox` (function, `tests/test_data_pipeline.py:44`) `def test_verify_python_ast_and_sandbox()`
- `test_write_jsonl_roundtrip` (function, `tests/test_data_pipeline.py:51`) `def test_write_jsonl_roundtrip(tmp_path)`
- `test_to_chat_row` (function, `tests/test_data_pipeline.py:58`) `def test_to_chat_row()`
- `normalize_text` (function, `topogpt3/data_pipeline.py:37`) `def normalize_text(text)`
- `content_hash` (function, `topogpt3/data_pipeline.py:44`) `def content_hash(text)`
- `quality_filter` (function, `topogpt3/data_pipeline.py:48`) `def quality_filter(text)`
- `dedup_stream` (function, `topogpt3/data_pipeline.py:66`) `def dedup_stream(rows, key_field, fuzzy)`
- `parse_jsonl` (function, `topogpt3/data_pipeline.py:85`) `def parse_jsonl(path)`
- `write_jsonl` (function, `topogpt3/data_pipeline.py:96`) `def write_jsonl(path, rows)`
- `TeacherClient` (class, `topogpt3/data_pipeline.py:107`) `class TeacherClient` - Minimal OpenAI-compatible chat client for teacher generation.
- `__init__` (method, `topogpt3/data_pipeline.py:110`) `def __init__(self, base_url, model, api_key, timeout, temperature, max_tokens)`
- `chat` (method, `topogpt3/data_pipeline.py:126`) `def chat(self, system, user)`
- `sandbox_python` (function, `topogpt3/data_pipeline.py:155`) `def sandbox_python(code, timeout)` - Run Python through the eval sandbox when available; AST-only fallback.
- `verify_completion` (function, `topogpt3/data_pipeline.py:171`) `def verify_completion(completion, mode)`
- `generate_teacher_rows` (function, `topogpt3/data_pipeline.py:195`) `def generate_teacher_rows(rows, teacher, domain, verify, max_keep)`
- `to_chat_row` (function, `topogpt3/data_pipeline.py:233`) `def to_chat_row(row)`
- `clean_stage` (function, `topogpt3/data_pipeline.py:242`) `def clean_stage(input_path, output_path)`

## Internal vs External Edges

- Internal resolved imports (EXTRACTED): 3
- Cross-boundary resolved imports (EXTRACTED): 2

## Connections

- [EXTRACTED] depends_on community 0 <-> 5 (strength 0.9): Extracted import edge crosses communities: eval/harness.py imports eval/sandbox.py.
- [EXTRACTED] depends_on community 3 <-> 5 (strength 0.9): Extracted import edge crosses communities: topogpt3/tools_agent.py imports eval/sandbox.py.
- [INFERRED] shares_context community 4 <-> 5 (strength 0.5): Inferred shared context (language py) with no import path between community 4 (topogpt3: jlens) and community 5 (eval: data_pipeline).
- [INFERRED] shares_context community 5 <-> 6 (strength 0.5): Inferred shared context (layer utility) with no import path between community 5 (eval: data_pipeline) and community 6 (orphans).
- [INFERRED] bridges community 2 <-> 5 (strength 0.4): Inferred cross-community bridge: tests/test_checkpoint_meta.py reaches tests/test_data_pipeline.py in 7 hops.
- [INFERRED] bridges community 5 <-> 1 (strength 0.4): Inferred cross-community bridge: tests/test_data_pipeline.py reaches topogpt3/hodge_cm.py in 7 hops.
- [INFERRED] bridges community 5 <-> 2 (strength 0.4): Inferred cross-community bridge: tests/test_data_pipeline.py reaches topogpt3/tokenizer_tools.py in 7 hops.

## Risks

- [taint high] `eval/harness.py` -> `eval/sandbox.py` via `subprocess` (1 hops)

## Open Questions

- What would break if the most connected file in eval: data_pipeline changed?
- Should eval: data_pipeline be split, given cohesion 0.60?

## Sources

- `eval/sandbox.py`
- `eval/sandbox_smoke.py`
- `tests/test_data_pipeline.py`
- `topogpt3/data_pipeline.py`