# Token Evaluations System

<figure><img src="/files/CfzPQtNhjtnZV72Jb4yL" alt=""><figcaption></figcaption></figure>

This documentation provides a comprehensive overview of how the AIQ system processes, evaluates, and manages cryptocurrency tokens across various stages. The system combines market data analysis, technical indicators, and AI-powered evaluations to provide insights on tokens and generate trading signals.

### System Overview

The AIQ platform follows these main workflows:

1. **Token Discovery and Intake**: Tokens are discovered through monitoring services, user searches, or direct submission
2. **Data Collection**: Multiple data fetchers gather information from various sources
3. **Evaluation Process**: Tokens undergo evaluation against configurable criteria
4. **Scoring and Decision Making**: AI models and rule-based checks determine token quality and trading signals
5. **Token Status Management**: Tokens are categorized based on evaluation results

### Core Evaluation Engine

#### Evaluation Types

The system supports three primary evaluation types:

1. **Status Evaluation** (`ConfigType.STATUS`): Evaluates token quality and viability
2. **Buy Evaluation** (`ConfigType.BUY`): Determines if a token is a good buying opportunity
3. **Sell Evaluation** (`ConfigType.SELL`): Analyzes if a token should be sold

#### Evaluation Entry Points

The main entry point for token evaluation is `evaluateTokenConfig`, which directs to the appropriate evaluation type:

```typescript
export const evaluateTokenConfig = async (
  token: Token,
  cutoffDate: Date,
  checkConfigType: ConfigType,
  userQuantId?: number,
) => {
  switch (checkConfigType) {
    case ConfigType.BUY:
      return evaluateBuy(token, cutoffDate, userQuantId);
    case ConfigType.SELL:
      return evaluateSell(token, cutoffDate);
    case ConfigType.STATUS:
      return evaluateStatus(token, cutoffDate);
    default:
      throw new Error(`Invalid check config type: ${checkConfigType}`);
  }
};
```

Each evaluation type follows a similar pattern but has specific processing logic for the evaluation results.

#### Core Evaluation Logic

The `evaluateTokenWithType` function provides the shared evaluation framework:

1. Sets up logging context
2. Initializes a `TokenDataFetcher` to gather data
3. Ensures required data is available
4. Calculates market cap for token categorization
5. Fetches appropriate evaluation configurations
6. Checks for recent evaluations to avoid redundant processing
7. Processes each pending evaluation configuration
8. Returns statistics on evaluation results

#### Check Processing

For each evaluation configuration, the system:

1. Runs all configured checks against gathered data
2. Calculates an internal score based on weighted results
3. Requests an AI-generated score using LLMs
4. Combines scores to determine the final result
5. Saves evaluation results to the database
6. Processes the results according to the evaluation type

#### Check Result Processing

The `processResults` function is central to the evaluation system:

```typescript
const processResults = async (
  configType: ConfigType,
  results: MergedCheckResult[],
  evalConfig: EvalConfig,
  token: Token,
  dataFetcher: TokenDataFetcher,
  marketCap: number,
  logger: Logger,
): Promise<EvaluationResult> => {
  // Calculate internal score based on weighted results
  // Request AI score from LLM
  // Combine scores
  // Determine pass/fail status
  // Return complete evaluation result
}
```

Key aspects of result processing:

1. **Weight Calculation**: Checks have weighted importance in the overall score
2. **Required Checks**: Some checks are required; failing them fails the entire evaluation
3. **AI Analysis**: The system requests AI-generated analysis for holistic assessment
4. **Score Combination**: Internal and AI scores are averaged for a final score
5. **Threshold Application**: Different thresholds apply based on evaluation type

#### Value Comparison Logic

The `compareValues` function implements the core comparison logic for checks:

```typescript
export function compareValues(
  value: number,
  { extraInfo, threshold, name }: CheckConfig,
): ComparisonResult {
  // Handle value validation
  // Process range-based comparisons
  // Apply specified comparison operator
  // Calculate percentage score
  // Return comparison result
}
```

The system supports several comparison modes:

* Range-based comparisons: Check if value falls within a specified range
* Greater-than-or-equal comparisons: Check if value ≥ threshold
* Less-than-or-equal comparisons: Check if value ≤ threshold

The `getPercScore` function converts raw values to percentage scores (0-100) based on the comparison mode, allowing for nuanced scoring rather than binary pass/fail assessment.

### Post-Evaluation Processing

Each evaluation type has specific result processing:

#### Status Evaluation Processing

For status evaluations:

* If passed, the token is queued for buy evaluation
* For test tokens, processing is limited to a single status approval

#### Buy Evaluation Processing

For buy evaluations:

* If passed, the token is queued for execution and status is set to ACTIVE
* Buy execution messages are sent to applicable users

#### Sell Evaluation Processing

For sell evaluations:

* If passed, sell execution messages are created for applicable users
* Sell reason is recorded as "bad performance evaluation"

### Result Persistence

The `saveResultToHistory` function stores evaluation results:

```typescript
const saveResultToHistory = async (
  configType: ConfigType,
  tokenId: number,
  result: EvaluationResult,
) => {
  // Determine evaluation result type
  // Create database record with all details
  // Track metrics for monitoring
}
```

This function:

1. Maps config type to evaluation result type
2. Creates a database record with full evaluation details
3. Includes check-level results for diagnostics
4. Records metrics for system monitoring

### Key Components

#### 1. Token Discovery and Processing

**New Token Calls (`newTokenCall.ts`)**

This service handles the creation of new tokens and token calls:

```javascript
export const newTokenCall = async (
  newToken: { contractAddress: string; ticker: string; name?: string },
  sourceType: SourceType,
  message?: string,
  sourceId?: number,
): Promise<BaseTokenData> => {
  // Log token discovery
  // Get or create token in database
  // Queue token for evaluation
  // Track metrics
  // Return token data
}
```

The function:

* Accepts token information (address, ticker, optional name)
* Records the source of the token (e.g., USER, BIRDEYE, DEXSCREENER)
* Creates the token record if it doesn't exist
* Queues token for evaluation
* Returns basic token data

**Processing Token Sources (`processTokenSources.ts`)**

This file contains functions for processing tokens from various sources:

* `processTokenSources`: Processes tokens from monitoring services
* `processBoostSources`: Processes tokens from DexScreener boosts

Both functions filter out already processed tokens to avoid duplication, then create new token calls for unique discoveries.

**Token Monitors**

Several monitoring services continuously discover new tokens:

1. **BirdEye Monitor** (`birdeyeMonitor.ts`): Monitors BirdEye API for:
   * New tokens
   * Trending tokens
   * High-liquidity tokens
2. **DexScreener Monitor** (`dexscreenerMonitor.ts`): Monitors DexScreener for:
   * Tokens with active boost campaigns
3. **Twitter Monitor** (`twitterMonitor.ts`): Monitors Twitter for:
   * Token mentions from selected accounts

#### 2. Data Collection and Management

**TokenDataFetcher (`TokenDataFetcher.ts`)**

This class handles fetching and caching data for token evaluation:

```javascript
export class TokenDataFetcher {
  private dataCache: TokenDataTypes = {} as TokenDataTypes;
  
  async init({
    skipOhlcv = false,
  }: { skipOhlcv?: boolean } = {}): Promise<void> {
    // Initialize cache with data from database
    // Fetch OHLCV data if needed
  }
  
  getData<K extends DataKey>(key: K): TokenDataTypes[K] {
    // Get data for specific key with validation
  }
}
```

The class:

* Fetches data from multiple sources (BirdEye, DexScreener, etc.)
* Caches responses to avoid redundant API calls
* Provides typed access to different data categories
* Implements data expiration to ensure freshness

**Candle Aggregation (`candleAggregator.ts`)**

This module processes price candle data for technical analysis:

```javascript
export const getLastHour1mCandlesAndPrior23Hours1hCandles = (
  tokenPrice: TokenPrice[],
) => {
  // Extract last hour of 1-minute candles
  // Aggregate remaining data into hourly candles
  // Sort in ascending time order
  // Return [lastHour1mCandles, prior23Hours1hCandles]
}
```

The function:

* Takes raw price data points
* Extracts the most recent hour as 1-minute candles for detailed analysis
* Aggregates the previous 23 hours as hourly candles
* Returns both arrays in time-ascending order

#### 3. Token Evaluation Process

**Core Evaluators**

The system uses two core evaluator classes:

1. **CoreStatusEvaluator** (`CoreStatusEvaluator.ts`): Base class for evaluating token status
2. **CoreTradeEvaluator** (`CoreTradeEvaluator.ts`): Base class for evaluating buy/sell signals

Both follow a similar pattern:

* Gather required data via `TokenDataFetcher`
* Run configured checks against the data
* Calculate internal score based on check results
* Request AI score via LLM
* Combine scores and determine final result
* Save evaluation results to database

**Token-Specific Evaluators**

Two specialized evaluator classes extend the core evaluators:

1. **TokenStatusEvaluator** (`TokenStatusEvaluator.ts`): Evaluates token quality
2. **TokenTradeEvaluator** (`TokenTradeEvaluator.ts`): Evaluates trading opportunities

Both classes:

* Load appropriate configuration based on market cap category
* Support both core (default) and degen (customized) configurations
* Override configurations based on user preferences
* Apply appropriate checks for each evaluation type

**Check Registry**

The system includes many individual check functions, such as:

* `narrative.ts`: Evaluates token narrative strength
* `priceTrendPrediction.ts`: Predicts price movement direction
* `buyToSellMaxRatio1h.ts`: Checks buy/sell volume ratio
* `liquidity.ts`: Evaluates token liquidity
* `marketRegime.ts`: Determines market conditions
* `priceChangePercent1h.ts`: Checks price change percentage
* `priceDrawdown.ts`: Evaluates price drops from highs

Each check:

* Takes a configuration and a `TokenDataFetcher` instance
* Extracts relevant data points
* Performs calculations
* Returns a `CheckResult` with score, value, and pass/fail status

#### 4. AI-Powered Analyses

The system uses AI models for complex analyses:

**Narrative Analysis (`getNarrative.ts`)**

Evaluates token narratives using LLMs:

```javascript
export const getNarrativeStrength = async (
  token: BaseTokenData,
  description?: string,
): Promise<ModelResult<ParsedScoreResult>> => {
  // Generate prompt
  // Get AI response
  // Parse result
  // Return narrative score and explanation
}
```

The function:

* Creates a prompt describing the token
* Sends the prompt to OpenAI API
* Parses the JSON response
* Returns a score (0-100) and explanation

**AI Prompts (`prompts.ts`)**

Contains prompt templates for AI services:

1. `getNarrativePromptMessages`: Generates prompts for evaluating token narratives
2. `getTokenStatusScorePromptMessages`: Generates prompts for overall token evaluation
3. `getPriceTrendPredictionPromptMessages`: Generates prompts for price trend analysis

Each template:

* Includes detailed system instructions
* Formats token data appropriately
* Provides examples to guide the AI response
* Specifies expected output format

#### 5. Token Status Management

**Token Status Updater (`tokenStatusUpdater.ts`)**

Manages token status transitions:

```javascript
export const tokenStatusUpdater = async (token: Token, status: TokenStatus) => {
  // Reset dormant counter for ACTIVE tokens
  // Increment dormant counter for DORMANT tokens
  // Transition to REJECTED when dormant threshold exceeded
  // Update token status in database
}
```

The function:

* Updates token status based on evaluation results
* Manages dormant counter for inactive tokens
* Automatically rejects tokens after multiple dormant evaluations

**Token Data Services (`tokenData.ts`)**

Provides functions for retrieving token data:

* `getTokenWithEvaluationResult`: Retrieves tokens with their evaluation results
* `getTokensByStatus`: Retrieves tokens filtered by status
* `getPriceSnapshot`: Retrieves token price snapshots
* `sortTokens`: Helper function for consistent token sorting

#### 6. Queue System (`queues.ts`)

The queue system orchestrates asynchronous processing:

```javascript
export class QueueService {
  private queues: Map<string, Queue> = new Map();
  
  // Helper methods for queue management
  
  addEvaluateToken = (data: QueueDefinitions["evaluateTokenInitDataFetching"]["data"]) => {
    // Add token to evaluation queue
  }
  
  // Methods for scheduling and periodic tasks
  
  setupSchedulers = async () => {
    // Set up recurring tasks
  }
}
```

The service:

* Creates and manages Bull MQ queues
* Provides methods for adding jobs to queues
* Sets up schedulers for periodic tasks
* Supports retry logic and error handling

Key queues include:

* `evaluateTokenInitDataFetching`: Initial evaluation of discovered tokens
* `evaluateTokenFetchedData`: Evaluation after data fetching
* `evaluateBuy`: Evaluation for buy signals
* `evaluateSell`: Evaluation for sell signals

#### 7. Token Search (`tokenSearcher.ts`)

Provides search functionality for tokens:

```javascript
export const search = async (
  params: SearchParams,
): Promise<TokenSearchResponse> => {
  // Search token by id, address, or symbol
  // Create new token if needed
  // Fetch token overview data
  // Return token with overview data
}
```

The function:

* Accepts search parameters (id, address, symbol)
* Attempts to find existing token
* Creates new token if not found and source exists
* Fetches overview data for context
* Returns token and overview information

#### 8. WebSockets Subscription Manager (`WebsocketsSubscriptionManager.ts`)

Manages real-time data subscriptions:

```javascript
export class WebsocketsSubscriptionManager {
  // Singleton pattern implementation
  
  async subscribeToTokenPriceFeed(token: BaseTokenData) {
    // Subscribe to token price feed
  }
  
  async unsubscribeFromTokenPriceFeed(token: BaseTokenData) {
    // Unsubscribe from token price feed
  }
}
```

The service:

* Manages WebSocket subscriptions via Redis pub/sub
* Allows subscribing to token price feeds
* Supports unsubscribing when monitoring ends

### Configuration Framework

The system uses a flexible configuration framework:

#### Market Cap Categories

Tokens are categorized by market cap:

* Nanocap: < $100K
* Microcap: < $1M
* Smallcap: < $10M
* Midcap: < $100M
* Largecap: < $1B
* Megacap: > $1B

#### Check Configuration

Each check is configurable with:

* Threshold: Target value to compare against
* Weight: Importance in overall score (0-100)
* Required: Whether failing this check automatically fails evaluation
* Disabled: Whether to skip this check
* Operator: How to compare value against threshold (GTE or LTE)
* Range: Optional min/max range for comparison

#### Core and Degen Modes

Two configuration modes:

* Core Mode: Standard configurations managed by system
* Degen Mode: User-customizable configurations for advanced users

### Key Workflows

#### Token Discovery Workflow:

1. Monitoring service finds new token
2. `newTokenCall` creates/updates token record
3. Token is queued for evaluation
4. Data fetchers gather token information
5. Evaluation determines token status

#### Token Evaluation Workflow:

1. Load appropriate configuration
2. Fetch required data
3. Run checks against data
4. Calculate internal score
5. Request AI score
6. Combine scores and determine result
7. Update token status
8. Save evaluation results

#### Trading Signal Workflow:

1. Token selected for evaluation
2. Technical and market checks performed
3. AI analysis requested for price trend
4. Buy/sell signals generated based on results
5. Signal results saved and potentially acted upon

### Error Handling and Resilience

The system implements several resilience features:

* Queue-based processing with retries
* Data caching to reduce API dependency
* Fallback mechanisms for missing data
* Error tracking and logging
* Special handling for unavailable data errors
* Tolerance for test tokens
* Metrics tracking for monitoring system health

### Database Models

Key database models include:

* `Token`: Core token information
* `TokenPrice`: Historical price data
* `DataFetcherResponse`: Cached responses from data sources
* `EvaluationResult`: Results of token evaluations
* `EvaluationResultDetails`: Detailed check results

### Getting Started for New Developers

1. Understand the core evaluation pipeline in `evaluators/index.ts`:
   * How configurations are loaded
   * How checks are executed
   * How results are processed
   * How AI integration works
2. Explore the check system:
   * How individual checks work
   * How checks are configured
   * How the comparison logic in `compareValues` works
   * How results are aggregated
3. Study the result processing functions:
   * `processStatusEvaluations`
   * `processBuyEvaluations`
   * `processSellEvaluations`
4. Examine how results are persisted:
   * `saveResultToHistory`
   * Metrics tracking

### Advanced Topics

Once familiar with the core system, explore:

1. Custom check development
2. Configuration management
3. AI prompt optimization
4. Performance tuning of data fetchers
5. Enhancing comparison operators and scoring methods

### Conclusion

The AIQ Evaluations and Token Processing system is a sophisticated platform combining market data analysis, technical indicators, and AI-powered evaluations. It provides a flexible, configurable framework for discovering, analyzing, and evaluating cryptocurrency tokens, supporting both automated and user-driven workflows. The system's core strengths lie in its modular check system, configurable evaluation pipeline, and integration of machine learning for holistic assessment.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.aiquant.fun/technical-docs/token-evaluations-system.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.