Demystifying AI API Pricing: Understanding Prompt, Completion, and Cache Token Costs

When integrating APIs for various AI Large Language Models (like OpenAI's GPT-4 or Anthropic's Claude 3.5), developers often encounter a variety of billing terms and prices across different consoles or third-party tools. In our daily development practices at `wiki.lib00.com`, we frequently see developers confused by API bills: Why do some dashboards show prices as $0, while official pricing is quite expensive? What exactly is the relationship between Input, Prompt, and Completion? This article will systematically explain the price differences and core terminology related to AI APIs. ## 1. Core Concept: What is a Token? AI models do not bill by character or word count, but rather by **Tokens**. A Token is the fundamental unit of text processing for a model. * **Conversion Rate:** 1,000 Tokens roughly equal 750 English words or 400-500 Chinese characters (depending on the specific tokenizer used). * **Billing Unit:** Official APIs typically use **1M Tokens** (one million tokens) as the standard billing unit. --- ## 2. Three Billing Dimensions and Terminology Equivalence Different vendors (like OpenAI, Anthropic, Google) or developer tools may use different wording in their UI, but they essentially revolve around three core dimensions. The following three sets of terms are **completely equivalent**: ### Dimension 1: The Input Side (Input = Prompt) * **Equivalence:** `Input Price` ≡ `Prompt Price` * **Definition:** All the text information you send to the AI, including System Prompts, conversation history, uploaded documents, and your current query. * **Cost Characteristic:** This is the cost incurred for the AI to "read" your request. It is typically **much cheaper** than output prices (usually a 1:3 to 1:5 ratio) because "reading context" consumes less compute power than "thinking and generating new content." * **Usage Habits:** Early OpenAI documentation mostly used **Prompt**, while Anthropic (Claude) and Google (Gemini) tend to use **Input**. ### Dimension 2: The Output Side (Output = Completion) * **Equivalence:** `Output Price` ≡ `Completion Price` * **Definition:** The final response content generated by the AI based on your instructions. * **Cost Characteristic:** This is the **most expensive part** of your AI bill. The model requires complex autoregressive probability calculations for every single Token it generates, leading to the highest GPU resource consumption. * **Usage Habits:** "Completion" is a technical term, as the essence of an LLM is to "complete" the subsequent characters based on the preceding text. For the sake of clarity, more and more platforms are simply calling it **Output**. ### Dimension 3: The Cache Side (Cached Input = Cache Price) * **Equivalence:** `Cached Input Price` ≡ `Cache Price` * **Definition:** The unit price for Tokens when you ask consecutive questions and a large amount of repetitive context in your Prompt (like the same long document or codebase) hits the system cache. * **Cost Characteristic:** Enjoys a **massive discount** (usually only 10% to 50% of the original input price). The AI reads the cached state directly without recalculating the Attention matrix. * **Usage Habits:** This is a recently popularized cost-reduction technology. Anthropic calls it **Context Caching**, and OpenAI calls it **Prompt Caching**. --- ## 3. Terminology Comparison Table Whenever you see these two sets of words in any API dashboard, you can treat them as identical: | Billing Stage | Term A (Common/Intuitive) | Term B (Technical/Traditional) | Actual Meaning | Cost Proportion | | :--- | :--- | :--- | :--- | :--- | | **In** | **Input** | **Prompt** | What you send to the AI | Lower | | **Out** | **Output** | **Completion** | What the AI replies to you | Highest | | **Store** | **Cached Input** | **Cache** | The "discounted" part of repeated context | Very Low | --- ## 4. Why might you see a $0.0000 price? Sometimes developers might see prices like this in certain clients: ```plaintext Prompt price: $0.0000/1M Completion price: $0.0000/1M Cache price: $0.0000/1M ``` While official pricing (e.g., Claude 3.5 Sonnet) might be: ```plaintext Input: $2.50 / 1M tokens Cached input: $0.25 / 1M tokens Output: $15.00 / 1M tokens ``` The reason for this massive discrepancy is: 1. **$0.0000 is often a placeholder or free resource:** It typically appears during free trials, when using locally deployed open-source models (like Llama 3 via Ollama), or as a default display when API management software hasn't been configured with a specific price table. 2. **$2.50/$15.00 is real commercial production pricing:** This represents the actual computational cost required to invoke top-tier closed-source models. --- ## Summary and Advice As `DP@lib00` often emphasizes, when developing AI applications, if you see all-zero prices, please ensure you have correctly configured your API Key or billing rules. If you are using official paid interfaces, **controlling the length of the Output is the key to saving money**, as that is the bulk of the bill. Concurrently, fully utilizing the **Cache** mechanism can drastically reduce Input costs for long-context conversations.