Project Context

POLYMARKET LAB — AI PROJECT MEMO

======================================================================
1. WHAT THIS PROJECT IS
======================================================================

Polymarket Lab is a separate internal engineering project for collecting,
storing, viewing and later analyzing data from Polymarket.

Project goals:
- collect Polymarket markets via API
- store them in our own MariaDB database
- store history of market snapshots
- run cron-friendly jobs
- provide a PHP web dashboard for internal viewing
- later add CLOB market data
- later build signals
- later add paper trading
- later add analytics and possibly AI-assisted analysis

This is not a marketing website.
This is an internal data/engineering lab.

======================================================================
2. MAIN PROJECT PRINCIPLE
======================================================================

Architecture first, code second.

Any new code must fit into the project architecture cleanly.
Do not write chaotic code.
Do not mix responsibilities between layers.

Before proposing implementation, always understand:
- where the file belongs
- why it belongs there
- who will call it
- what it should return
- what layer it belongs to

======================================================================
3. PROJECT CONTEXT FROM USER
======================================================================

Important project rules explicitly requested by the user:

1. SQL rule:
Whenever proposing tables or columns for MySQL / MariaDB,
always explain every column:
- purpose
- values it stores
- meaning in project logic

Additionally, when appropriate, use COMMENT on table columns directly in SQL.

2. Architecture rule:
The user wants to design project structure and expandable architecture carefully first,
so the project does not become confusing later.

This rule must be preserved in future answers.

======================================================================
4. CURRENT PROJECT STRUCTURE
======================================================================

Python backend:
- app/clients/        external API access only
- app/repositories/   database access only
- app/services/       business logic
- app/jobs/           cron-friendly entry points
- app/utils/          common helpers/utilities
- app/models/         schemas / internal models

SQL:
- sql/                migrations / schema files

Web:
- web/public/         public PHP entry points
- web/public/api/     internal web API endpoints
- web/includes/       config, DB, layout, helpers
- web/public/assets/  CSS / JS / static assets

Runtime:
- var/logs/           application logs

======================================================================
5. PYTHON LAYER RULES
======================================================================

5.1 clients
Files in app/clients:
- work only with external APIs
- do not write directly to DB
- do not contain business orchestration
- return raw or lightly normalized data

Examples:
- polymarket_gamma.py
- polymarket_clob.py

5.2 repositories
Files in app/repositories:
- work only with DB
- contain SQL operations
- do not call external APIs
- do not perform business orchestration

Examples:
- upsert_market(...)
- insert_snapshot(...)
- create_job_log(...)

5.3 services
Files in app/services:
- contain business logic
- connect clients + repositories
- normalize data
- decide how to store and process data

5.4 jobs
Files in app/jobs:
- are cron-friendly entry points
- should be thin wrappers around services
- should not contain heavy business logic
- should:
  - create start log
  - run service
  - finish log
  - handle exceptions properly

======================================================================
6. SQL RULES
======================================================================

IMPORTANT PROJECT RULE:

Whenever proposing:
- a new table
- a new column
- a schema change
- an index

You must always explain EACH column:
- purpose of the column
- what values it stores
- what it means in the project logic
- whether it can be NULL
- typical expected values

Additionally:
- use COMMENT on tables when appropriate
- use COMMENT on columns when appropriate

Do not give raw CREATE TABLE without explaining the meaning of fields.

======================================================================
7. FILE HEADER RULE
======================================================================

Important files should contain a short header comment explaining:
- file path
- purpose
- what it does
- who calls it
- how to run it
- important notes

This is useful for:
- humans
- future maintenance
- AI analysis

======================================================================
8. LOGGING RULES
======================================================================

Project uses a simplified logging approach.

Main log file:
- var/logs/app.log

It stores:
- job start
- job finish
- errors
- API requests
- processing summaries
- important service steps

jobs_log table:
- is NOT a full technical log
- stores only short execution summaries:
  - job name
  - status
  - message
  - rows processed
  - timestamps

Do not overcomplicate logging without a real need.

======================================================================
9. CRON RULES
======================================================================

The project is intentionally cron-friendly.

Correct approach:
- separate job files
- no single infinite loop process unless really needed
- each job should be runnable:
  - manually
  - via cron

Example:
- python -m app.jobs.collect_markets
- docker compose exec -T polymarket_lab python -m app.jobs.collect_markets

Jobs should remain small and predictable.

======================================================================
10. WEB / PHP RULES
======================================================================

The PHP dashboard is an internal technical dashboard.

It should be:
- light
- compact
- tabular
- informative
- minimalistic
- easy to scan

The web UI should help understand:
- project data
- DB structure
- field names
- Russian explanation of DB fields

Do not turn it into a flashy public UI.

======================================================================
11. DO NOT DO THESE THINGS
======================================================================

Do not:
- mix client/service/repository layers
- put business logic into jobs
- put DB writes into API clients
- overengineer too early
- add unnecessary abstractions
- expose secrets from .env
- break working architecture for cosmetic reasons

======================================================================
12. WHAT IS ALREADY IMPLEMENTED
======================================================================

Already implemented:
- separate Docker project
- shared MariaDB usage
- Gamma API collector
- markets table
- market_snapshots table
- jobs_log table
- unified app log
- PHP dashboard
- project context tool

Working core files include:
- app/config.py
- app/db.py
- app/clients/polymarket_gamma.py
- app/repositories/*
- app/services/collector_service.py
- app/jobs/collect_markets.py
- app/utils/logger.py
- main PHP dashboard pages
- project context files

======================================================================
13. WHAT EXISTS AS PLACEHOLDERS
======================================================================

Some files may exist as architectural placeholders and still be empty.
That is intentional.

Typical placeholders:
- app/clients/polymarket_clob.py
- app/jobs/analyze_markets.py
- app/jobs/run_paper_trading.py
- app/models/*
- app/services/analyzer_service.py
- app/services/signal_service.py
- app/services/trader_service.py
- app/utils/dt.py
- app/utils/parsing.py
- sql/*.sql (if not yet filled)
- README.md
- health.php

AI must distinguish between:
- implemented production logic
- placeholders prepared for future work

======================================================================
14. CURRENT PRIORITY
======================================================================

Priority order should generally be:

1. Fix and store SQL schema files in sql/
2. Add CLOB client
3. Add collect_clob_snapshots job
4. Collect best_bid / best_ask / spread / midpoint
5. Then signals
6. Then paper trading
7. Then analytics

Do not jump too far ahead unless specifically requested.

======================================================================
15. RUNTIME PATHS
======================================================================

Important path distinction:

Host machine path:
- /srv/sites/polymarket_lab

Web/PHP runtime path inside container mount:
- /var/www/polymarket_lab

AI should keep this distinction in mind and not confuse host path with runtime path.

======================================================================
16. HOW TO ANSWER WELL FOR THIS PROJECT
======================================================================

Good answer pattern for this project:

First:
- explain where the new file/table/module belongs in architecture
- explain why it belongs there

Then:
- give implementation

If SQL is involved:
- explain every column carefully

If a new module is proposed:
- explain:
  - purpose
  - caller
  - input
  - output
  - role in architecture

======================================================================
17. BEST CONTEXT PACKAGE FOR AI
======================================================================

Best package to send to AI for this project includes:

1. AI Project Memo
2. Current Task
3. Project Tree
4. File Path List
5. Database Schema Dump
6. Table Relationships
7. Implemented vs Placeholder
8. Files Summary
9. File Contents

If the full context is too long for a model,
split it into multiple sequential parts.
Parts must preserve whole file blocks.

======================================================================
18. CURRENT TASK SHOULD ALWAYS BE EXPLICIT
======================================================================

When working with AI on this project, always state the current task clearly.

Examples:
- now we are fixing SQL migrations
- now we are designing CLOB layer
- now we are building collect_clob_snapshots.py
- now we are improving dashboard
- now we are not touching paper trading yet

This improves relevance and reduces architectural drift.

======================================================================
19. SUMMARY
======================================================================

Polymarket Lab is a modular internal engineering project.
The main value is not fast random code, but a clean maintainable system.

Good solutions for this project are:
- readable
- layered
- predictable
- expandable
- aligned with architecture
- careful with SQL meaning
- careful with runtime paths
- useful for both humans and AI