xAI logo

xAI Software Engineer System Design Questions

20 practice questions for xAI Software Engineer interviews

xAI software engineer interviews cover algorithms, data structures, system design, and coding problems drawn from real interview rounds.

Software Engineer Backend Engineer Frontend Engineer Full Stack Engineer Mobile Engineer Data Engineer Data Scientist ML Engineer DevOps Engineer DevOps Engineer Product Manager SRE Security Engineer Engineering Manager Data Analyst UX/UI Designer QA Engineer

No verified questions yet for xAI.

system design Medium stack queue #1

1. Design GrokRequestBatcher — deterministic, model-aware inference batching


Background
xAI's Grok inference service needs to combine compatible requests into batches while balancing throughput and waiting time. An anonymous xAI interview report described completing a Python function for batching language-model requests; this exercise adapts that reported task into a self-contained class design. The scheduling rules below are exercise specifications, not claims about xAI's production implementation.
Requirements
1. Maintain pending requests in insertion order. Each request has an ID, an opaque model key, a positive token reservation, an arrival time, and a time-to-live. A batch must contain requests for exactly one model and must respect both the configured token and request-count limits.
2. Every public operation receives now_ms. Before performing that operation, remove every request whose expiration time is at most now_ms. Expiration is arrival_ms + ttl_ms. Time is nondecreasing across calls, and expiration takes precedence over selection or cancellation.
3. submit returns false if its ID is already pending after expiration cleanup; otherwise it appends the request and returns true. Duplicate submissions must not update the existing request. An ID may be reused after its request expires, is canceled, or is dispatched.
4. A model is ready if any of these conditions holds after cleanup: its total pending token reservations are at least max_tokens; its pending request count is at least max_requests; or its oldest pending request has waited at least max_wait_ms. Among ready models, choose the model whose oldest pending request was inserted earliest. Insertion order, not model-name order, breaks equal-arrival-time ties.
5. next_batch removes and returns exactly one batch for the selected model. Take the longest nonempty prefix of that model's insertion-ordered queue that fits both limits. Stop at the first request that would exceed a limit: do not skip it to fit smaller requests behind it. If no model is ready, return an empty list. A readiness threshold is based on the entire model queue, even when only a short prefix fits.
6. Support cancellation by ID and inspection of pending IDs in global insertion order. All operations are single-threaded; no background timer, inference execution, networking, or persistence is required.
Class API
  • GrokRequestBatcher(max_tokens: int, max_requests: int, max_wait_ms: int) — construct an empty batcher with fixed limits.

  • submit(request_id: str, model: str, tokens: int, now_ms: int, ttl_ms: int) -> bool — clean up expired requests, then attempt to enqueue a request.

  • cancel(request_id: str, now_ms: int) -> bool — clean up expired requests, then remove the specified pending request; return whether it existed.

  • next_batch(now_ms: int) -> list[str] — clean up expired requests, select one ready model, and return dispatched IDs in that model's queue order.

  • pending(now_ms: int) -> list[str] — clean up expired requests and return all remaining IDs in global insertion order.


Example 1:
Input sequence, using opaque model key text:
{
  "config": {"max_tokens": 10, "max_requests": 3, "max_wait_ms": 50},
  "calls": [
    ["submit", "a", "text", 6, 0, 100],
    ["submit", "b", "text", 5, 1, 100],
    ["submit", "c", "text", 4, 2, 100],
    ["next_batch", 2],
    ["next_batch", 3],
    ["next_batch", 51],
    ["pending", 51]
  ]
}

Output, one result per call:
[true, true, true, ["a"], [], ["b", "c"], []]

Explanation: At time 2, text is ready because its total reservation is 6 + 5 + 4 = 15 >= 10 and its count is 3. The batch contains only a: adding b would use 11 tokens, and skipping b to include c is forbidden. At time 3, the remaining total is 9 and the oldest request has waited only 2 ms. At time 51, b has waited exactly 50 ms, so b and c dispatch together.
Example 2:
Input sequence:
{
  "config": {"max_tokens": 8, "max_requests": 2, "max_wait_ms": 20},
  "calls": [
    ["submit", "x", "text", 3, 0, 10],
    ["submit", "y", "vision", 4, 1, 100],
    ["submit", "z", "vision", 4, 2, 100],
    ["next_batch", 2],
    ["submit", "w", "text", 5, 3, 100],
    ["cancel", "w", 4],
    ["pending", 10],
    ["cancel", "x", 10],
    ["submit", "x", "text", 8, 10, 5],
    ["submit", "x", "text", 1, 10, 100],
    ["next_batch", 10]
  ]
}

Output:
[true, true, true, ["y", "z"], true, true, [], false, true, false, ["x"]]

Explanation: At time 2, only vision is ready: its two requests reserve 4 + 4 = 8 tokens. The older text request does not block it. Request w is canceled, and the original x expires exactly at time 10. Its ID can then be reused. The second submission of the new x is rejected without changing its 8-token reservation, so it dispatches immediately.
Constraints
  • 1 <= max_tokens <= 1,000,000, 1 <= max_requests <= 1,000, and 0 <= max_wait_ms <= 1,000,000.

  • At most 2,000 requests are pending at once, with at most 10,000 public method calls per instance. An O(n) operation is acceptable for this exercise.

  • IDs and model keys are nonempty strings of at most 100 characters. Every submitted reservation satisfies 1 <= tokens <= max_tokens; 1 <= ttl_ms <= 1,000,000.

  • Timestamps are nonnegative integers and are nondecreasing across all calls, including unsuccessful submissions and cancellations. Inputs satisfy these bounds.
Want to practise this one?
Create a free account to open the full question and practise xAI questions with AI.
Sign up free
system design Medium api design #2

2. Design UserSessionManager — A class to manage user sessions in an online platform

1. Background: The UserSessionManager class is critical for xAI to handle user sessions securely and efficiently. It requires proper management of user logins, session timeouts, and session validation to ensure a consistent user experience across devices.
2. Requirements:
1. The class should allow user logins and logouts.
2. Handle session timeouts after a specified duration.
3. Validate existing sessions.
4. Store session data securely.
3. Class API:
- login(user_id: str) -> str: Logs in a user and returns the session ID.
- logout(session_id: str) -> None: Logs out a user by invalidating their session ID.
- is_valid_session(session_id: str) -> bool: Checks if the session ID is valid.
- get_session_data(session_id: str) -> dict: Retrieves the session data associated with a session ID.
4. Example 1:
- Input: login('user123') → Output: session_id_abc
- Explanation: User user123 successfully logs in and receives a session ID.
5. Constraints:
- Max 1000 active sessions.
- Session timeout duration: 30 minutes.
- Session IDs are unique.
- Thread-safe operations.
system design Senior api design #3

3. [OA] API Design — Constructing xAI's Machine Learning Model API

As xAI expands its machine learning capabilities, we need a robust API to manage model training, predictions, and evaluations. This API should handle requests efficiently and return results in a unified format.
Problem Statement: Design an API that allows users to submit training data, request model training, and retrieve predictions. The API should support the following operations: POST /train, GET /predict, and GET /status. Ensure to consider versioning and error handling in your design.
Example Operations:
  • POST /train: Accepts a JSON payload with data and returns a 201 status with a task ID.

  • GET /predict: Accepts a task ID and returns predictions based on trained data.

  • GET /status: Checks the status of the training job based on task ID.

Constraints:
  • Ensure the API can handle a minimum of 1000 concurrent requests with low latency.

  • Should support flexible data formats (CSV, JSON).
system design Hard caching #4

4. [OA] Caching — Designing xAI's Custom In-Memory Cache

To enhance performance for xAI's machine learning algorithms, we need a fast, in-memory caching system to store frequently accessed data. This system should allow for efficient data retrieval and support for automatic eviction of old data.
Problem Statement: Design a custom cache that supports the following operations: put(key: int, value: int): void and get(key: int): int. When the cache reaches its capacity, it should invalidate the least recently used (LRU) item.
  • class LRUCache:

  • def __init__(self, capacity: int): Initializes the cache with a positive size capacity.

  • def get(self, key: int) -> int: Returns the value of the key if the key exists, otherwise returns -1.

  • def put(self, key: int, value: int) -> None: Updates or adds the value if the key is not present. When the cache reaches its capacity, it should invalidate the least recently used item before adding the new item.

Example 1:
Input:
LRUCache cache = new LRUCache(2);
cache.put(1, 1);
cache.put(2, 2);
cache.get(1); // returns 1
cache.put(3, 3); // evicts key 2
cache.get(2); // returns -1 (not found)
Example 2:
Input:
LRUCache cache = new LRUCache(1);
cache.put(1, 1);
cache.get(1); // returns 1
cache.put(2, 2); // evicts key 1
cache.get(1); // returns -1 (not found)
Constraints:
  • 1 <= capacity <= 3000

  • 0 <= key <= 10000

  • 0 <= value <= 10^9.
system design Hard caching #5

5. [OA] Caching — Implement a simple in-memory cache for xAI's AI model responses

To accelerate response times from our AI models, xAI needs to implement a caching mechanism for the frequently accessed data.
Design and implement a simple LRU Cache class that supports the following operations:
  • get(key: int) -> int: Retrieve the value of the key if the key exists in the cache, otherwise return -1.

  • put(key: int, value: int) -> None: Update the value of the key if the key exists. If the key does not exist, add the key-value pair to the cache. If the cache reaches its capacity, it should invalidate the least recently used item before inserting a new item.


Example Method Signatures:
  • def get(self, key: int) -> int

  • def put(self, key: int, value: int) -> None


Example 1:
Input: cache = LRUCache(2), cache.put(1, 1), cache.put(2, 2), cache.get(1)
Output: 1
Example 2:
Input: cache.put(3, 3), cache.get(2)
Output: -1
Explanation: The least recently used key (2) was removed because the cache reached its capacity.
Constraints:
  • capacity is at most 3000.

Related xAI Software Engineer interview prep

Start practicing xAI questions

Sign up for free to access walkthroughs, AI-generated questions, and more.

Get Started Free