trading/tradebot
Watch
1
0
Fork
You've already forked tradebot
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat: automated futures contract rollover #1

Merged
derfenix merged 18 commits from feat/futures-rollover into master 2026-07-20 19:47:41 +00:00
Conversation 2 Commits 18 Files changed 43 +4740 -59
derfenix commented 2026-07-19 22:19:17 +00:00
Owner
Copy link

Summary

  • Add per-trader futures rollover: daily check at 10:00 UTC within days_before of LastTradeDate, switch to the next contract in the same series via FuturesNavigator.
  • Roll is 1:1 under suspension: short limit orders on ordinary days (unfilled → postpone), market on the final safe day; book via ApplyRoll, update config FIGI, recreate trader, resume.
  • No persistent roll state — restart re-evaluates the live position. Failures after the position is touched → ASDegraded + Notifier. Details in docs/rollover.md.

Test plan

  • direnv exec . go test ./...
  • direnv exec . golangci-lint run
  • Paper: enable rollover near expiry, confirm flat switch and 1:1 roll with seeded quotes
  • Confirm degrade + alert path when a leg fails after fills
  • Confirm postponed roll resumes trading and retries next day
## Summary - Add per-trader futures rollover: daily check at 10:00 UTC within `days_before` of `LastTradeDate`, switch to the next contract in the same series via `FuturesNavigator`. - Roll is **1:1** under suspension: short limit orders on ordinary days (unfilled → postpone), market on the final safe day; book via `ApplyRoll`, update config FIGI, recreate trader, resume. - No persistent roll state — restart re-evaluates the live position. Failures after the position is touched → `ASDegraded` + `Notifier`. Details in `docs/rollover.md`. ## Test plan - [ ] `direnv exec . go test ./...` - [ ] `direnv exec . golangci-lint run` - [ ] Paper: enable `rollover` near expiry, confirm flat switch and 1:1 roll with seeded quotes - [ ] Confirm degrade + alert path when a leg fails after fills - [ ] Confirm postponed roll resumes trading and retries next day
- Instrument.LastTradeDate (zero for non-futures)
- FuturesNavigator port with NextFutures/PrevFutures and sentinel errors
  ErrNotFutures/ErrNoNextFutures/ErrNoPrevFutures
- FuturesSeriesPrefix/SameFuturesSeries helpers keeping rollover within
  one contract series (never mix SILV and SILVM)
- TraderConfig.Rollover (enabled, days_before) with default and validation
- config.Updater interface, satisfied by yamlconfig.Adapter
- rollover section in config.yaml.example
- State model: FIGIs, direction, lots progress, phase, close price,
  order refs for idempotent recovery, daily 10:00 UTC check dedup
- Store interface with ErrNotFound/ErrInvalidState sentinels
- localfs.RolloverStore: atomic temp+rename JSON writes, mutex-guarded
Add Info.NextFutures/PrevFutures returning fully resolved
domain.Instrument for the adjacent contract in the same series
(same basic asset + FIGI series prefix, so SILV never mixes with
SILVM). Contracts are ordered by LastTradeDate ascending with
strict comparison for next/prev selection.

The raw futures catalog (INSTRUMENT_STATUS_ALL) is cached
process-wide with a 24h TTL behind a mutex; per-series filtering
is applied on every read and returns a fresh slice.

LoadInstrumentInfo now fills Instrument.LastTradeDate from
Future.GetLastTradeDate() when resolving futures.

Depends on domain branch: Instrument.LastTradeDate,
FuturesSeriesPrefix, ErrNotFutures/ErrNoNextFutures/ErrNoPrevFutures.
Hooks skipped: package does not compile until domain integration.
Introduce app/rollover package with pure calculation logic for futures
rollover:

- Quote/PairedQuote top-of-book value types with explicit validation
  errors (missing, invalid, crossed)
- DirectionalSpread: long = nextAsk - oldBid, short = oldAsk - nextBid
- ComputeBaseline: mean/stddev (Welford) over up to the latest 100
  synchronized 5-minute paired observations, minimum 30 samples
- EvaluateSpread: favorable when spread <= mean - stddev and the
  improvement over the mean covers total estimated commission
- MarginNeutralLots: floor(oldLots*oldMargin/newMargin) capped by
  maxLots and available margin; never increases source exposure
- Table-driven parallel tests for all of the above
Trader keeps its engine after Start and exposes Send for queue commands
and Snapshot with FIGI, direction, lots, LastTradeDate, state, equity
and margin fields for rollover sizing. Start accepts WithDeferredReady
so a caller can take over the ASReady transition. DataSource bypasses
the cache TTL when the requested FIGI differs from the cached one and
gains an explicit RefreshInstrument for forced upstream reloads.
Introduce domain RolloverQuotes and RolloverOrderPlacer ports, an explicit
PlaceRolloverOrder engine command (no LotSizer), paper simulation, and
tinvest orderbook/history/orders adapters scoped to the rollover flow.
Service runs a per-trader goroutine with a daily check at 10:00 UTC
(catch-up on start), recovers persisted pending rolls idempotently,
evaluates roll spread favorability with a final-safe-day override,
sizes the next contract margin-neutrally and executes the transfer
sequentially: limit close, limit open with fill-timeout cancellation,
market fallback on the final safe day only. The trader config FIGI is
updated and the trader recreated only after the full transfer.

Dependencies are narrow interfaces (TraderOps, QuoteSource,
OrderExecutor, ConfigUpdater, RecreateTrader) so the package needs no
concrete trader or adapter types yet.
When ServiceConfig.Commission is zero, the spread favorability check now
uses a conservative estimate derived from the current contract prices
(DefaultCommissionRate x (oldAsk + nextAsk)) instead of treating the
roll as commission-free.
- Application requires config.Updater and builds a localfs.RolloverStore
  (same cache root as accounts) when any trader has rollover enabled;
  rollover configs are normalized (default days_before) and validated on load
- Start launches a rollover.Service per rollover-enabled trader after the
  traders themselves; app.Provider now requires FuturesNavigator and
  RolloverQuotes
- bridge adapters: TraderOps over engine commands targeting the current
  trader, QuoteSource over domain.RolloverQuotes with paper quote seeding,
  OrderExecutor over domain.RolloverOrderPlacer (paper simulator or
  account-scoped tinvest orders) with idempotent request IDs,
  ConfigUpdater over yamlconfig UpdateTraderConfig
- trader recreation runs under the application mutex; traders with a
  pending rollover state start with deferred Ready
Rollover orders bypass account bookkeeping, so a recreated paper trader
loaded the stale old-contract lots from the account cache and attached
them to the new instrument. finalize now applies the executed roll to
the account exactly once (persisted State.Booked flag) via a close+open
OperationPlaced pair through the engine queue: the old position is
closed at the roll close price (realizing PnL) and the new-contract
position is opened at the persisted open fill price. Live accounts
still reload the broker portfolio on recreation.

Also persist the broker-assigned order ID on rollover order refs
(OrderRef.BrokerID) so recovery in a new process can cancel the order
remainder at the broker instead of falling back to a no-op empty fill.
evaluate() read the live position and sized the roll before executeRoll
suspended the trader, so the strategy loop could change the position
between sizing and execution. The daily check now suspends the trader
first (after the cheap expiry/chain pre-checks), captures the
authoritative position and sizing under suspension, and resumes trading
on every path that leaves no pending roll behind (skip, postpone, early
failure, no-position contract switch). Recovery via continueRoll now
owns the suspension for persisted rolls instead of executeRoll.
- bookRoll no longer pretends success when persisting the Booked flag
  fails after a successful ApplyRoll: the error is returned and the
  booking is remembered in memory so a same-process retry does not book
  the roll twice
- finalize persists a Switched flag after switching the contract, so a
  retry never recreates the trader again; deleting the pending state is
  best effort and never blocks the resume
- ClosePrice/OpenPrice are maintained as volume-weighted averages across
  partial fills (new OldLotsClosed counter as the close-side weight), so
  booking a partially limit-filled roll with a market remainder realizes
  the correct PnL
- extend domain.RolloverOrderPlacer with GetRolloverOrderStateByRequestID:
  tinvest resolves through OrderIdType REQUEST and maps the broker
  "order not found" (50005) to ErrRolloverOrderNotFound; the paper
  simulator resolves through its request-ID index
- orderExecutor.Cancel no longer reports an empty fill for an unknown
  submitted order: without a persisted broker ID it resolves the order
  through the request ID and treats only a confirmed not-found as
  "never accepted"; any other failure aborts the reconciliation so a
  possibly live order is never replaced blindly
- paper rollover order states are persisted per account through a new
  file-backed localfs.PaperOrderStore and restored on environment
  creation, so restart recovery can resolve and cancel simulated orders
  placed by a previous process
Two crash-safety fixes in the roll phase machine:

- Reconciling a submitted order after a failed cancel re-applied the
  cumulative broker fill on top of the already applied partial fill,
  double-counting lots and corrupting the VWAP prices. OrderRef now
  persists the applied fill progress (FilledLots/FillPrice) and
  applyOrderFill folds only the not-yet-applied delta into the state,
  which is correct for partial-then-cancel-failure, partial-then-more
  fill and full-fill reports.

- bookRoll ran ApplyRoll before persisting the Booked flag, so a crash
  between the two re-booked the roll on restart. The booking intent is
  now persisted first and ApplyRoll is convergent: the bridge is a
  no-op when the account already carries the roll target position, and
  a recovered Booked state re-runs it once to converge.
The convergence check required the account position to carry the new
FIGI, but OperationPlaced never switched the position instrument: after
a successful booking the account still showed the old contract, so a
restart with Booked=true re-booked the roll (double PnL).

- Position adopts the operation instrument when opening from flat
- ApplyRoll resolves the full next-contract instrument for the open leg
  (bare FIGI fallback when the lookup fails)
- rollApplied also recognizes a booked position relabeled to the old
  FIGI by the datasource force refresh (every lot at the roll open
  price)
- regression tests: repeated ApplyRoll is a no-op and never doubles
  FixedPnL; datasource keeps booked lots when the configured old FIGI
  forces an instrument refresh
Three durability fixes in the roll finalize/recovery path:

- rollApplied no longer treats an old-FIGI position whose lots all carry
  the roll open price as already booked: prices can collide with a
  genuine live position, silently skipping the close/reopen while the
  contract switch still proceeds. Only the new-contract FIGI (plus
  direction and lots) proves booking; any old-FIGI position is closed
  and reopened onto the new contract by the convergent ApplyRoll.

- ensureSwitched persists the Switched intent (after the idempotent
  config FIGI update) BEFORE recreating the trader, so a store failure
  after a successful recreation can no longer recreate the trader twice
  on retry. A failed recreation is retried without repeating the config
  update; after a restart the re-run is a benign same-config restart.

- placeLimitTracked/placeMarketTracked no longer mark a tracked order
  terminal with an empty fill when the submission returns an error: the
  order may still be live at the broker (timeout after acceptance). The
  order is reconciled through the executor (request-ID lookup, empty
  fill only on broker-confirmed not-found); when the reconciliation
  fails the ref stays Submitted so recovery resolves it before placing
  anything new.
Drop the phase machine, spread baseline, margin-neutral sizing, and
exactly-once order recovery. Daily check rolls 1:1 with short limit
(postpone) or final-day market, degrades+notifies on hard failures.
derfenix commented 2026-07-19 22:22:46 +00:00
Author
Owner
Copy link

Ревью MR #1: feat: automated futures contract rollover

1. RequestID — случайный UUID, идемпотентность не работает через рестарты

orderExecutor.place() генерирует свежий uuid.NewString() как RequestID при каждом вызове. Если процесс упадёт между PostOrder и обработкой ответа, после перезапуска rollover запустит place() с новым UUID — и создаст дублирующий ордер на бирже.

Фактический риск невысокий (окно узкое, трейдер suspended), но если хочется идеальной идемпотентности — RequestID должен выводиться из детерминированных входных данных (FIGI + direction + lots + kind). Например, хеш от этих полей.

2. contractSwitcher.Switch — контекст может быть отменён

В цепочке evaluate → roll → finalize → Switch используется родительский ctx. Если во время rollover придёт SIGHUP (hot-reload) или SIGTERM, контекст отменится, и updateTraderFIGI / recreateTrader могут не успеть сохранить конфиг с новым FIGI. Рекомендую использовать context.WithoutCancel(ctx) или отдельный context.Background() для операций записи конфига.

3. futuresCatalogTTL = 24h — может пропустить свежий контракт в день проверки

Если каталог загружен, а Tinkoff добавил новый контракт в течение дня, nextCheckDelay отложит проверку до завтра, а каталог обновится только через 24ч от первой загрузки. Предлагаю форсировать инвалидацию каталога при каждом успешном ежедневном чеке в evaluate(), а не полагаться только на TTL.

4. PairCandleQuotes — map по time.Time может не сматчиться

nextByTime использует time.Time как ключ map. Ты нормализуешь через .UTC(), но у HistoricCandle из Tinkoff время может быть с наносекундами или без. Явное округление до минут/интервала не помешало бы для надёжности.

5. TestFuturesCatalogTTL мутирует shared state

Тест форсирует инвалидацию через прямое изменение info.futuresCache.loadedAt. Хотя тесты по умолчанию не параллелят пакеты, модификация shared глобального sharedFuturesCatalog может проявиться при go test -count=1 ./... в соседних тестах. Рекомендую сохранять/восстанавливать состояние каталога (или обернуть в setup/teardown).

6. normalizeRolloverConfigs — days_before: 0 становится дефолтом

Если в конфиге явно указано days_before: 0, нормализация превращает его в 3. Стоит иметь в виду — intentional zero не поддерживается. Можно через указатель если когда-нибудь понадобится.

В целом архитектура чистая, тесты отличные, документация есть. Ничего критического. Мержить можно, замечания опциональны.

## Ревью MR #1: `feat: automated futures contract rollover` ### 1. RequestID — случайный UUID, идемпотентность не работает через рестарты `orderExecutor.place()` генерирует свежий `uuid.NewString()` как `RequestID` при каждом вызове. Если процесс упадёт между `PostOrder` и обработкой ответа, после перезапуска rollover запустит `place()` с новым UUID — и создаст дублирующий ордер на бирже. Фактический риск невысокий (окно узкое, трейдер suspended), но если хочется идеальной идемпотентности — `RequestID` должен выводиться из детерминированных входных данных (FIGI + direction + lots + kind). Например, хеш от этих полей. ### 2. `contractSwitcher.Switch` — контекст может быть отменён В цепочке `evaluate → roll → finalize → Switch` используется родительский `ctx`. Если во время rollover придёт `SIGHUP` (hot-reload) или `SIGTERM`, контекст отменится, и `updateTraderFIGI` / `recreateTrader` могут не успеть сохранить конфиг с новым FIGI. Рекомендую использовать `context.WithoutCancel(ctx)` или отдельный `context.Background()` для операций записи конфига. ### 3. `futuresCatalogTTL = 24h` — может пропустить свежий контракт в день проверки Если каталог загружен, а Tinkoff добавил новый контракт в течение дня, `nextCheckDelay` отложит проверку до завтра, а каталог обновится только через 24ч от первой загрузки. Предлагаю форсировать инвалидацию каталога при каждом успешном ежедневном чеке в `evaluate()`, а не полагаться только на TTL. ### 4. `PairCandleQuotes` — map по `time.Time` может не сматчиться `nextByTime` использует `time.Time` как ключ map. Ты нормализуешь через `.UTC()`, но у `HistoricCandle` из Tinkoff время может быть с наносекундами или без. Явное округление до минут/интервала не помешало бы для надёжности. ### 5. `TestFuturesCatalogTTL` мутирует shared state Тест форсирует инвалидацию через прямое изменение `info.futuresCache.loadedAt`. Хотя тесты по умолчанию не параллелят пакеты, модификация shared глобального `sharedFuturesCatalog` может проявиться при `go test -count=1 ./...` в соседних тестах. Рекомендую сохранять/восстанавливать состояние каталога (или обернуть в setup/teardown). ### 6. `normalizeRolloverConfigs` — `days_before: 0` становится дефолтом Если в конфиге явно указано `days_before: 0`, нормализация превращает его в `3`. Стоит иметь в виду — intentional zero не поддерживается. Можно через указатель если когда-нибудь понадобится. --- В целом архитектура чистая, тесты отличные, документация есть. Ничего критического. Мержить можно, замечания опциональны.
derfenix commented 2026-07-19 22:32:14 +00:00
Author
Owner
Copy link

По замечаниям из ревью:

#2 (ctx cancel на Switch) — починено: context.WithoutCancel в ContractSwitcher (wire) и в finalize / switchAndResume, чтобы запись конфига, recreate и resume не обрывались на SIGTERM/SIGHUP mid-roll.

#4 (PairCandleQuotes) — оказалось мёртвым кодом после упрощения (history больше не используется сервисом). Убрано вместе с PairedQuoteHistory, PairedTopOfBook и WithDeferredReady.

#1, #3, #5, #6 — оставляем как опциональные follow-up (idempotent RequestID, invalidate futures catalog, тест TTL, days_before: 0).

Пуш: refactor(rollover): drop unused quote history and harden switch ctx.

По замечаниям из ревью: **#2 (ctx cancel на Switch)** — починено: `context.WithoutCancel` в `ContractSwitcher` (wire) и в `finalize` / `switchAndResume`, чтобы запись конфига, recreate и resume не обрывались на SIGTERM/SIGHUP mid-roll. **#4 (PairCandleQuotes)** — оказалось мёртвым кодом после упрощения (history больше не используется сервисом). Убрано вместе с `PairedQuoteHistory`, `PairedTopOfBook` и `WithDeferredReady`. **#1, #3, #5, #6** — оставляем как опциональные follow-up (idempotent RequestID, invalidate futures catalog, тест TTL, `days_before: 0`). Пуш: `refactor(rollover): drop unused quote history and harden switch ctx`.
Remove PairedQuoteHistory/PairCandleQuotes and WithDeferredReady; keep
only TopOfBook for pricing. Use WithoutCancel for finalize/switch so
config write and recreate survive parent cancellation.
derfenix merged commit 7c451a25ac into master 2026-07-20 19:47:41 +00:00
derfenix deleted branch feat/futures-rollover 2026-07-20 19:47:41 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
trading/tradebot!1
No description provided.