Guides
Troubleshooting
Symptom, cause and fix for agents that will not connect, calls that fail or time out, refused requests, and webhooks that do not arrive.
Start with the two built-in tools, Check connection and the Playground, then find your symptom below. Every message in code style is the exact text the service returns. If you still cannot solve it, the last section lists what to include in a support request.
Two tools to use first
Check connection
On every project page, the Check connection panel finds out why an agent or a client cannot reach the project. Press Run checks. Optionally paste a key first: it is compared on our side, never stored or logged, and a verdict is only given for keys of your own project. It tests, in order:
| Finding | Level | What it means and what to do |
|---|---|---|
This project is suspended. | fail | Agents and clients are refused until it is reinstated. The reason is shown. Contact us |
The monthly allowance on the <plan> plan is used up. | fail | On Free, calls are refused until the 1st (UTC) or until you move to a larger plan. On paid plans it is a warning: calls are still served and counted |
Most of the monthly allowance is used. | warn | You are past 80%. On Free, calls are refused at 100% |
There is no agent key. | fail | Create an agent (or full) key and give it to the service that registers your tools |
No agent is connected. | fail | If a key has been used before, the agent has run: check it is running now, can reach this service outbound over HTTPS, and is not on a platform that freezes idle connections. If no key was ever used: check SPACEFIX_API_KEY and SPACEFIX_SERVICE_URL in your service, then start it |
<n> agent(s) is/are connected. | ok | Good |
The agent is connected but has registered no tools. | warn | Register tools before you connect, and read the agent's log for a rejected registration |
Every tool is disabled. | warn | Clients see an empty tool list. Re-enable them under Tools |
<n> of <m> tools are visible to clients. | ok | Good |
Every client of this project is limited to reading: <r> of <v> tools read. / That key is limited to reading: <r> of <v> tools read. | ok | Client access is Read only, or the pasted key is read-only. Only the tools that read are visible |
The same, with 0 of <v> tools read | warn | No tool is marked as reading, so read-only clients see an empty list. Set readOnlyHint on the tools that only read, or mark them Read under Tools. Unmarked tools count as writes |
There is no client key. | warn | MCP clients need a client or full key. Agent keys are refused on the MCP endpoint |
Clients are limited to <n> allowed address(es). | warn | A client calling from anywhere else is refused with "not on the allow-list" |
That is an agent key. | warn | It registers tools. MCP clients need a client or full key |
That key is valid for this project. | ok | The key is fine. Look elsewhere |
That key is not recognised. | fail | Look for a truncated copy or a stray space. A lost key cannot be recovered, only replaced |
That key belongs to a different project. | fail | Use a key created in this project, or the other project's endpoint |
That key was revoked. / That key has expired. | fail | Create a new key |
It cannot test the network from your agent's or client's side. It tells you what the service knows.
The Playground
The Playground (sidebar, under Build) and the Try a tool panel on each project page run one of your connected agent's tools through the same path an MCP client takes: the same schema check and the same agent, with no client configuration in between. So:
- Works in the Playground, fails in your client means the problem is in the client's configuration, key or network, not in your agent. Compare against Connect a client.
- Fails in the Playground too means look at the agent or the tool's code. The result shows the error your handler returned.
The Playground is not counted toward your allowance, but it is limited to 10 runs a week per account (The playground allows <n> runs a week and they are used up. It resets <date> UTC.) and 30 a minute (Too many test calls. Try again in a minute.). It also says That tool is not registered right now. when no connected agent has it, That tool is disabled. Enable it to try it., Arguments must be a JSON object., and The arguments do not match the tool's schema. with the reasons listed.
The agent will not connect
Your agent keeps retrying, backing off from 1 second up to 30, and logs each attempt. The Node.js SDK logs [spacefix] <message>; retrying in <n>ms, where the message is API key was rejected (<status>). Check SPACEFIX_API_KEY for a 401 or 403, Registration was rejected (400): <body> for a 400, and Connect failed: <status> <body> for anything else. The Go SDK logs Connect failed: <status> <body> for every refusal. When the connection works you see [spacefix] connected with <n> tool(s).
The table is keyed by what the gateway answered, which is the same for every SDK.
| Gateway answer | Cause | Fix |
|---|---|---|
401 Missing API key. | No Authorization header: SPACEFIX_API_KEY is unset or empty | Set SPACEFIX_API_KEY to an agent or full key |
401 Invalid API key. | Mistyped, truncated, revoked or expired key, or a key from another deployment | Create a new key on the project page and check for whitespace. Run Check connection with the key pasted |
403 This is a client key. Agents connect with an agent key. | You gave the agent a client key | Create an agent (or full) key |
429 Too many failed authentication attempts. | Many wrong keys from this address in a minute, often a revoked key in a retry loop | Fix the key. The limit lifts by itself after Retry-After seconds. Valid keys are never blocked by it |
429 Too many agent connections: the <plan> plan allows <n>. | More replicas than the plan allows for one project | Scale down, or use a plan with more connections. Check the Agents page for stale ones |
429 Rate limit exceeded: <n> requests per minute. | Connecting too often, usually a crash loop | Fix the crash loop. The SDK honours Retry-After |
400 Too many tools: limit is <n>. | More tools than the plan allows | Register fewer |
400 Invalid tool name "<name>". | Tool names may use only letters, digits, _ and -, 1 to 64 characters | Rename the tool |
400 Schema for "<tool>" exceeds <n> bytes. | One tool's schema is too large | Simplify it |
400 Schema for "<tool>" is not valid JSON Schema: <reason> | The schema does not compile | Fix the schema. The reason says where |
400 Invalid prompt "<name>". | A prompt name outside the same rule, or arguments that is not a list | Rename it or fix it |
400 Invalid resource ..., Duplicate resource "<uri>"., Invalid name for resource "<uri>"., Too many resources: limit is 1000. | A resource without a uri, two with the same uri, a non-text name, or more than 1,000 | Fix the registration |
400 "tools" must be an array. | A malformed registration | Use an SDK rather than hand-built requests |
415 Content-Type must be application/json. | A hand-built request without the JSON content type | Send Content-Type: application/json |
403 with code -32003 This project is suspended: <reason> | The project is suspended | Contact us |
503 Service is shutting down. | The gateway is restarting | The SDK reconnects by itself |
A network error such as fetch failed, ENOTFOUND, a TLS error or a timeout | Wrong SPACEFIX_SERVICE_URL, no outbound HTTPS from the agent's network, or a proxy in the way | Check the URL. If SPACEFIX_SERVICE_URL is unset the SDK uses its built-in default address, which is probably not your gateway. Test with curl <SPACEFIX_SERVICE_URL>/health from the same host: it answers {"ok":true} |
| It connects, then drops again and again | See the next table |
It connects, then drops
| Cause | Fix |
|---|---|
| The key was revoked, or the project's keys rotated, while connected. The stream ends at once | Give the agent the new key |
The key expired while connected. The stream ends, and the reconnect gets 401 Invalid API key. | Create a new key before expiry (you get emails at 14 and 3 days) and restart the agent with it |
| The project or account was deleted, or the project suspended | Check Activity |
| Running on request-scoped serverless (Lambda, Vercel functions, standard Cloudflare Workers) or Cloud Run without always-on CPU | The platform freezes the process. Run the agent as an always-on worker |
| A proxy or load balancer on your agent's outbound path buffers or times out long streams | The gateway sends a heartbeat every 25 seconds and asks proxies not to buffer. Set the proxy's idle timeout above that, or exempt this host |
The process exits: a crash, or you called disconnect() | Check your own logs |
A project shows Offline or Disabled
| Where you see it | Meaning | Fix |
|---|---|---|
| Project pill Offline, Dashboard status Offline, the top-bar chip says "Offline" | No agent is connected | See The agent will not connect |
Project pill <n> agent(s) disabled, Dashboard status Disabled | Agents are connected but every one has been switched off on the Agents page. Clients get an error | Open Agents and enable at least one. The switch belongs to the connection: an agent that restarts comes back enabled |
| Status Errors | An agent is connected and calls in about the last minute failed | Open Logs filtered to Failed. See Observability |
| An agent you expect is missing on the Agents page | It is not connected, or it disconnected. Agent ids exist only while connected | Check its logs and the key it uses |
A client gets "No agent connected"
The client sees HTTP 503 with JSON-RPC code -32001.
| Message | Cause | Fix |
|---|---|---|
No agent connected for server "<project-id>". | No agent is connected to that project, or the gateway restarted and your agent has not reconnected yet | Start the agent, or wait for it to reconnect. Check you are using the right project id, and that the agent's key belongs to this project |
Every agent for server "<project-id>" is disabled. | Agents are connected but all disabled | Enable one on the Agents page |
This happens on every request, including initialize and tools/list, so a client cannot even list tools. The failed request is neither logged nor counted.
A related message in a tool's error text is No connected agent serves tool "<name>". It means agents are connected but none of them registered that tool, resource or prompt (for example during a rolling deploy). Logs labels it No agent connected.
Tools are missing, or "Tool not found"
A client reports Tool <name> not found (JSON-RPC -32602), or tools/list is shorter than you expect.
| Cause | How to tell | Fix |
|---|---|---|
| The tool is disabled by you | The project page shows it as disabled. Check connection warns Every tool is disabled. if all are | Enable it under Tools |
| The client's key has a tool allow-list that excludes it | The keys table shows the allow-list | Use a key with a matching pattern, or create one |
| The client is read-only and the tool does not read | The keys table shows Read only, Project settings shows Client access: Read only, or the OAuth token has only mcp:read. Check connection counts the tools that read | Set readOnlyHint: true on tools that only read, or mark them Read with Edit under Tools. Unlabelled tools count as writes. Or use a key that reads and writes |
| The agent never registered it | The project page lists only what the running agent registered | Register it before connecting, then restart the agent |
| The agent was rejected on connect (invalid name, bad schema, too many tools) | The agent's log shows the refusal | See The agent will not connect |
| A different replica has an older tool set | During a rolling deploy the list comes from the longest-connected enabled agent | Wait for the rollout to finish |
| The client cached an old list | Other clients see the tool | Refresh or restart the client. The gateway sends no change notifications |
| You are on the wrong project's URL or key | 403 Forbidden - agent identity mismatch. | Match the URL and the key |
| Using OAuth | A token for another project, or no server_id claim | See OAuth problems |
Arguments that fail the tool's schema come back as a tool result with isError: true, for example Input validation error: Invalid arguments for tool echo: data must have required property 'text', not as "not found". They never reach your handler and are not counted. The gateway ignores the pattern and format keywords when validating, so enforce those in your handler.
Calls time out
A call fails with the tool result Request for tool "<name>" timed out after <ms>ms. (or resource "<uri>", prompt "<name>"). The call is counted, because it reached your agent.
| Cause | How to tell | Fix |
|---|---|---|
| The handler is slow | Logs: the call's Handler time is close to its duration | Make it faster. For slow work, return a job id and add a second tool to poll it |
| The agent's process is saturated | Dashboard: Event loop delay is high, CPU is near 100% (needs telemetry) | Add replicas, or move heavy work off the event loop |
| The project's call timeout is lower than your handler needs | Project settings, Call timeout (seconds) | Raise it, up to 30 seconds. Blank means 30 |
| The client gave up first | The client's own timeout is shorter than ours | Raise the client's timeout, or speed up the tool |
| The agent could not return the result | A network failure posting it is logged as [spacefix] failed to return result for "<name>": .... A proxy blocking POST /agent/result looks the same | Check egress rules |
| The result was too large | The limit is 4 MB. The Node.js SDK answers with Result too large for the gateway (limit 4194304 bytes) instead. An agent that does not do that times out | Return less data |
| The agent dropped mid-call | The tool result reads Agent disconnected before returning a result. (or Key revoked., Key rotated., Project deleted., Account deleted., Project suspended.) | Fix why it disconnected. The call is not retried |
| The handler never settles (a promise that never resolves) | Time is all handler time, every time | Fix the handler: always return or throw |
What the HTTP statuses mean
On the MCP endpoint and the agent endpoints. The full list is in Errors.
| Status | Where | Meaning | Fix |
|---|---|---|---|
400 | Agent connect | The registration was invalid. The message says what | See the connect table |
400 | MCP endpoint | The request body was not valid JSON, or not a JSON-RPC message | Send a valid JSON-RPC request |
401 | Both | Missing API key. (no header) or Invalid API key. (unknown, revoked or expired) | Send Authorization: Bearer <key>, or replace the key |
403 | Both | Wrong key kind (This is an agent key. MCP clients connect with a client key. and the reverse), Forbidden - agent identity mismatch., project suspended (-32003), or address not on the allow-list (-32004) | Use the right key, URL and address |
405 | MCP endpoint | GET and DELETE are not supported: Method not allowed. | Use POST |
406 | MCP endpoint | Not Acceptable: Client must accept both application/json and text/event-stream | Send Accept: application/json, text/event-stream |
413 | Both | Body larger than 4 MB (512 KB for a registration) | Send less |
415 | Both | Unsupported Media Type: Content-Type must be application/json on the MCP endpoint, Content-Type must be application/json. on the agent endpoints | Set the content type |
429 | Both | A rate limit (Rate limit exceeded: <n> requests per minute.), too many failed keys, too many agent connections, or the monthly allowance (-32002) | Wait for Retry-After. See Allowance used up |
503 | Both | No agent connected, or every agent disabled (-32001), or Service is shutting down. during a restart | See above |
A 401 or 403 carrying {"error": "..."} comes from the gateway's own checks. A JSON-RPC error object with a negative code means the request got far enough to be rejected by a rule: -32001 no agent, -32002 allowance, -32003 suspended, -32004 allow-list, -32602 bad parameters or an unknown tool.
Allowance used up
429 with Monthly allowance of <n> calls on the <plan> plan is used up. It resets on the 1st (UTC), or upgrade the plan. and code -32002.
| Cause | Fix |
|---|---|
| You are on the Free plan and have used your calls this month | Wait for the 1st (00:00 UTC), or ask us for a larger plan. Listing tools still works |
| The count looks too high | The Usage page's This month bar and the Billing & plan page show what was counted. Timeouts and agent errors count. Refused calls, validation failures, cache hits and Playground runs do not. A client in a loop may be calling far more than you think: check Usage, Busiest tools |
| You expected to be over but are not blocked | Paid plans are not blocked past the allowance. Calls are counted as overage |
See Plans and limits.
Results differ between calls
Calls are round-robined across your agent's replicas, and each replica is a separate process with separate memory. If a tool keeps state in memory (a cache, a counter, a session, an uploaded file), consecutive calls land on different replicas and see different state. Other causes:
| Cause | Fix |
|---|---|
| State held in process memory | Keep it in your database or a shared cache |
| Replicas running different versions (mid-deploy) | Wait for the rollout. Ship handlers before exposing them |
| Replicas with different configuration or data (a stale local cache, a different environment) | Make them identical |
| The gateway cached a result (see below) | Turn the cache off for that tool |
To check, open the Agents page: each replica is listed separately with its own call counts and last error.
Stale descriptions and tool lists
| Symptom | Cause | Fix |
|---|---|---|
| You changed a tool's description in code and clients still show the old one | An agent registers its tools when it connects. Until the process restarts, we have the old text | Restart or redeploy the agent |
| The old description persists after a redeploy | A description override set in the dashboard replaces whatever the agent declares | Clear the override in the tool's controls |
| Some clients show a new description, others an old one | Clients cache tools/list | Refresh or restart the client |
| A just-changed override or disable takes a while to apply | Settings are cached for up to a minute | Wait a minute |
| The list shows a tool missing from your newest code | The list comes from one replica (the longest-connected enabled one). During a rolling deploy it is the old one | Wait for the rollout |
Cache surprises
If a tool has a cache time (0 to 3,600 seconds) in its controls, the gateway answers repeated identical calls without asking your agent.
| Symptom | Cause | Fix |
|---|---|---|
| A tool returns old data | A cached result is being served until the time expires | Lower or remove the cache time. Saving a tool's settings clears the project's cached results |
| You deployed a fix but the old answer persists | An agent re-registering does not clear the cache | Change a tool's settings (any change clears the project's entries), or wait |
| Two users see the same result | One cached result is shared by every client of the project. The arguments are the only key (their order is ignored) | Do not cache tools whose answer depends on who is asking |
| The call does not appear in Logs and is not counted | A cache hit never reaches your agent | Expected |
| An error was not cached | Errors are never cached, only successful results | Expected |
| A large result is never cached | Results over 256 KiB are not stored | Expected |
| The cache is empty after a gateway restart | It is held in memory | Expected |
OAuth problems
These apply only if OAuth is turned on for your deployment. See Security.
| Symptom | Cause | Fix |
|---|---|---|
A static client key is rejected with 401 | With OAuth on, the MCP endpoint accepts access tokens only | Use the OAuth sign-in flow |
| The client cannot discover where to sign in | The 401 challenge or the discovery document is wrong | Contact us |
Token has no "server_id" claim. | The identity provider does not put the project id in the token | The provider must add the claim |
403 Forbidden - agent identity mismatch. | The token's server_id is not the project in the URL | Get a token for the right project |
403 insufficient_scope, The token needs one of these scopes: … | The token has none of the scopes the gateway accepts (by default mcp:write, mcp:invoke or mcp:read) | Request one of them: mcp:write to read and write, mcp:read to read only |
| Signed in with OAuth but some tools are missing | The token has only mcp:read, so it sees only the tools that read | Request mcp:write, or label the tools that read (see above) |
401 after it worked yesterday | The access token expired. There is no revocation check, so expiry is the only end | Let the client refresh |
Signature, issuer or audience errors in the 401 description | The token is not from the configured issuer, or its audience is wrong | Fix the identity provider's audience setting |
Discovery answers 503 Authorization Server metadata is unavailable. | We cannot reach the identity provider's metadata | Contact us |
The client wants a client_id and offers nowhere to put it | The identity provider does not support Dynamic Client Registration | Pre-register the client, or enable it |
| Opaque (non-JWT) tokens | Not supported | Configure the identity provider to issue JWT access tokens |
Webhook failures
On the Webhooks page, a webhook's last delivery shows ok, failed (<status>) or failed: <error>. The rules are in Alerts and webhooks.
| Symptom | Cause | Fix |
|---|---|---|
Cannot save the URL: The URL must start with https://., That host is not reachable from the internet., That address is not reachable from the internet., Put credentials in the signature check, not the URL. | The URL is not allowed | Use a public https:// URL with no embedded credentials |
At most 5 webhooks per account. | The limit | Delete one |
Last delivery: getaddrinfo ENOTFOUND <host> | The hostname does not resolve | Fix the DNS name |
Last delivery: Resolved to a private address. | The name points at a private address | Use a public address |
Last delivery: Timed out. | Your endpoint took over 5 seconds | Answer first, work after |
Last delivery: failed (4xx) | Your endpoint refused it, often a signature check or authentication in front of the route. A 4xx other than 429 is not retried | Fix the endpoint |
Last delivery: failed (5xx) | Your endpoint errored. It was retried twice, after 10 seconds and 2 minutes | Fix the endpoint |
failed (301) or failed (302) | Redirects are not followed | Use the final URL |
switched off after 20 failed deliveries | 20 deliveries in a row failed | Fix the endpoint, then press Re-enable. Events during the gap are not replayed |
| Your signature check always fails | You used parsed-then-re-serialised JSON instead of the raw bytes, or the wrong secret (use the whole whsec_... string), or a proxy altered the body, or your clock is more than your tolerance away | Verify against the exact raw body. See the code samples |
| Send test works but real events never arrive | You did not subscribe to those events, the webhook is off, or the event did not occur. agent.connected and agent.disconnected fire only when a project goes between zero and one agent | Check the event ticks on the webhook. Tick none to receive all |
| You lost the secret | It is shown once | Delete the webhook and add it again |
Email is not arriving
This applies to verification, password-reset and email-change links, and to the alerts. Check the obvious first: the address on the account (Settings, Profile), your spam folder, and give it a few minutes. You can ask for another verification link once a minute from the banner on Settings.
| Symptom | Cause | Fix |
|---|---|---|
You cannot create projects or keys: Confirm your email address first. We sent a link when you signed up; you can ask for another in the banner. | Verification is required and you have not confirmed | Confirm, or press Resend link |
| Alerts never arrive | You turned that kind off under Settings, Email notifications, or the condition has not held yet (10 minutes offline, 80% of calls, 14 or 3 days before expiry) | Check Settings |
A link says That link has expired or was already used. | Links work once. Verification and email-change links last 24 hours, password-reset links last 1 hour | Ask for a new one |
If none of that explains it, contact us with the address and the time you expected the message.
What to include in a support request
Include enough to find your calls without pasting any secret.
- Never send a key or a webhook secret. Say its label and kind instead.
- The project id and project name, and your plan.
- The time, in UTC, and whether it is constant or intermittent.
- For a failed call: the Request id and Trace id from its row on Logs (open the row), or the call's trace from Traces. If the call was refused and never logged, send the exact HTTP status and JSON body the client got.
- The client and its version (Claude Code, Cursor, VS Code, your own code), and the exact configuration with the key removed.
- Your agent's side: the SDK and its version, the lines starting
[spacefix]from its log, and where it runs (container, VM, serverless). - What Check connection reports (copy the findings), and whether the Playground works for the same tool.
- Anything that changed just before: a deploy, a rotated key, a plan change.
Next steps
- Connect a client: the configuration for every client.
- Observability: Logs, Traces and failure types.
- Errors: every error with its status, code and fix.
- Alerts and webhooks