Migrating from OpenAI to open-source models: a practical checklist

Introduction

If you are reading this, you have likely hit one of the common pain points with OpenAI: rising costs, model deprecations, or simply the desire to evaluate alternatives that give you more control over your stack. The good news is that the ecosystem has matured. Many open-source model providers now expose an OpenAI-compatible API, meaning you can swap the backend without rewriting your application logic.

This checklist walks through the practical steps of migrating from OpenAI to an open-source model API. It assumes you are already using the OpenAI Python or curl SDK and want to switch to a provider like TokenShop, which offers pay-as-you-go access to models such as Qwen3 32B, DeepSeek V3.2, and GLM-4.6. The goal is to keep your code changes minimal while verifying that the new model meets your quality, latency, and cost requirements.

1. Verify API compatibility before touching code

The first rule of migration is: do not change your code until you have confirmed the new endpoint speaks the same language. Most open-source API services, including TokenShop, use the exact same /v1/chat/completions endpoint structure as OpenAI. This means your existing openai Python library or curl commands will work with a simple base URL swap.

Checklist item:

• Obtain an API key from your new provider.
• Test a basic chat completion using curl with the new base URL.

curl https://tokshop.xyz/v1/chat/completions \
  -H "Authorization: Bearer YOUR_NEW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "alibaba/qwen-3-32b",
    "messages": [{"role": "user", "content": "Say hello in one word."}]
  }'

If you get a valid JSON response with a choices array, you are ready to proceed. If you see a 402 error, it means your prepaid balance is too low — most providers require at least a small deposit (TokenShop, for example, gives $0.50 trial credit on registration and requires a balance above $0.001 to serve requests).

2. Update your SDK client with a one-liner change

Once the raw API works, update your Python code. If you are using the official openai Python package, you only need to change the base_url and api_key.

from openai import OpenAI

client = OpenAI(
    base_url="https://tokshop.xyz/v1",
    api_key="your-api-key-here"
)

response = client.chat.completions.create(
    model="deepseek/deepseek-v3.2",
    messages=[{"role": "user", "content": "What is the capital of France?"}]
)

print(response.choices[0].message.content)

Checklist item:

• Replace api_key and base_url in your client initialization.
• Ensure your environment variables are updated (e.g., OPENAI_API_KEY → your new key, or use a separate variable).
• Run a single test prompt that mirrors a production use case.

Note that the model name string will differ. For example, gpt-4o-mini becomes alibaba/qwen-3-32b or deepseek/deepseek-v3.2. Keep a mapping table in your config file to avoid hardcoding.

3. Compare context windows and pricing models

One of the biggest differences between OpenAI and open-source providers is the pricing structure. OpenAI charges per token with a premium on output tokens. Open-source APIs often have different ratios and significantly lower absolute costs.

Model	Input price (per 1M tokens)	Output price (per 1M tokens)	Context window
Qwen3 32B	$0.30	$0.90	131,072
DeepSeek V3.2	$0.40	$0.80	131,072
GLM-4.6	$0.80	$2.40	202,752

As of recent reports, GPT-4o-mini costs $0.15/$0.60 per million tokens, so some open-source models are competitive on input but may be pricier on output. However, models like DeepSeek V3.2 offer a generous 131K context window at a low output cost, which can be a win for long-document tasks.

Checklist item:

• Estimate your monthly token usage (input vs. output).
• Compare costs using the provider’s pricing page (e.g., TokenShop pricing).
• Check if your typical prompt fits within the model’s context limit. GLM-4.6’s 202K window is particularly useful for codebases or long conversations.

4. Test for quality regressions with a benchmark set

Open-source models are not drop-in replacements in terms of output quality. A model that scores well on benchmarks may still fail on your specific task. Build a small test set of 10–20 prompts that cover your core use cases: reasoning, instruction following, structured output, and safety.

Checklist item:

• Run the same prompt set against your current OpenAI model and the candidate open-source model.
• Compare outputs manually or using a simple scoring rubric (e.g., correctness, verbosity, formatting).
• Pay special attention to edge cases: long context, multi-turn conversations, and system prompt adherence.

For example, DeepSeek V3.2 tends to be strong on code generation and reasoning, while Qwen3 32B is often praised for instruction following in Chinese and English. GLM-4.6 excels at long-context retrieval. Choose based on your workload.

5. Handle billing and error states gracefully

Open-source API providers often use prepaid billing models. TokenShop, for instance, operates on a micro-USD ledger. If your balance drops below $0.001, the API returns an HTTP 402 status. Your application must handle this gracefully — retry with a fallback model or notify the user.

try:
    response = client.chat.completions.create(...)
except openai.APIStatusError as e:
    if e.status_code == 402:
        print("Insufficient balance. Top up at provider dashboard.")
        # Optionally fallback to a free or cached response
    else:
        raise

Checklist item:

• Add a 402 handler in your API call wrapper.
• Set up low-balance alerts (e.g., check your ledger via the provider’s dashboard or API).
• Consider a hybrid approach: use an open-source model for high-volume tasks and OpenAI for critical ones.

6. Review data privacy and compliance

One reason developers migrate to open-source APIs is data control. TokenShop’s data policy states that only metering metadata is stored; prompt contents are not persisted. This is a significant advantage over some closed APIs that may log prompts for model improvement.

Checklist item:

• Read the provider’s data policy. Confirm that prompts are not logged or used for training.
• If you handle sensitive data, verify that the API does not cache responses.
• Check if the provider offers a dedicated endpoint or private deployment (most open-source API services do not, but you can self-host the models if needed).

Conclusion

Migrating from OpenAI to an open-source model API is not a flip of a switch — it is a systematic process. The steps above give you a repeatable checklist: verify API compatibility, update your SDK client with a one-liner, compare pricing and context windows, test for quality regressions, handle billing errors, and review data privacy.

The key takeaway is that OpenAI-compatible APIs make the technical migration nearly trivial. The real work lies in understanding the trade-offs between models and ensuring your application handles the new cost and error profiles. Start with a single non-critical endpoint, run it for a week, and expand from there. For pricing details and model availability, see the TokenShop pricing page or the API documentation.