Creating a Polling API with Spring Boot and Scheduled Tasks

A polling API lets a client ask regularly whether a long-running operation has finished. This pattern suits report generation, payment verification, document processing and external integrations where an immediate response is impractical. Spring Boot provides the REST layer, while scheduled tasks can check pending work and update its status in the background.

For an Australian application, the design should also account for mobile users, variable network quality and traffic across Sydney, Melbourne, Brisbane and regional areas. Clear status responses reduce unnecessary requests, while careful handling of personal information supports obligations under Australia’s Privacy Act and the Australian Privacy Principles.

Design the polling contract

A useful workflow begins when a client creates a task. The server returns 202 Accepted and a resource identifier rather than keeping the connection open. The client can then request /api/jobs/{id} at a controlled interval until the job is complete, failed or expired.

A response should expose stable fields such as id, status, createdAt, updatedAt, result and errorCode. Typical states include PENDING, PROCESSING, COMPLETED, FAILED and EXPIRED. Avoid returning internal exception messages or sensitive payment details through the status endpoint.

{
  "id": "8f32c8d1",
  "status": "PROCESSING",
  "updatedAt": "2025-03-08T10:15:00+11:00"
}

Model jobs with persistent state

A database record is safer than an in-memory queue when the application runs across multiple instances. A JPA entity can store the current state, retry count, next attempt time and a small reference to the eventual result.

@Entity
public class PollJob {
    @Id
    private UUID id;

    @Enumerated(EnumType.STRING)
    private JobStatus status;

    private int attempts;
    private Instant nextAttemptAt;
    private Instant updatedAt;
}

Use a repository query to select eligible work rather than loading every record into memory. Store timestamps as UTC with Instant, then format them for users in the relevant Australian timezone. This avoids confusion when daylight saving changes between AEST and AEDT.

Run scheduled background processing

Enable scheduling in the application configuration and use a modest fixed delay. The scheduler should claim a small batch, process each job, then update its state. A fixed delay prevents the next cycle from starting immediately after the previous trigger if processing takes longer than expected.

@SpringBootApplication
@EnableScheduling
public class Application { }

@Scheduled(fixedDelayString = "${jobs.poll-delay-ms:5000}")
@Transactional
public void processPendingJobs() {
    repository.findReadyJobs(Instant.now(), PageRequest.of(0, 50))
              .forEach(this::process);
}

The task should be idempotent. If a network timeout occurs after an external payment gateway accepted a request, repeating the operation must not create a second charge. Use an idempotency key based on the job ID and persist the external reference returned by the provider.

Expose a clean REST endpoint

A controller can create the job and return a polling location. The client should receive a Retry-After value or equivalent guidance so it does not poll every few milliseconds.

@PostMapping("/jobs")
ResponseEntity<JobResponse> create() {
    PollJob job = service.createJob();
    URI location = URI.create("/api/jobs/" + job.getId());
    return ResponseEntity.accepted()
            .location(location)
            .body(JobResponse.from(job));
}

@GetMapping("/jobs/{id}")
JobResponse get(@PathVariable UUID id) {
    return service.findVisibleJob(id)
            .map(JobResponse::from)
            .orElseThrow(() -> new ResponseStatusException(
                    HttpStatus.NOT_FOUND));
}

Return 404 when the job does not exist or does not belong to the authenticated user. For a completed job, return the result or a short-lived download URL. This is particularly useful for mobile customers using public transport or patchy regional connectivity, where a small status response is preferable to repeatedly downloading a large document.

Control load, security and concurrency

Polling frequency must be bounded on both sides. Apply authentication, per-user rate limits and maximum job lifetimes. Conditional requests with ETag or Last-Modified let the API return 304 Not Modified when the status has not changed, reducing bandwidth and database pressure.

A scheduled method may execute simultaneously on several application nodes. Use a database lock, an atomic status update or a distributed scheduler such as ShedLock. A simple state transition can ensure that only one worker claims a PENDING row:

UPDATE poll_job
SET status = 'PROCESSING', updated_at = CURRENT_TIMESTAMP
WHERE id = ? AND status = 'PENDING';

Check the affected row count before processing. Record structured logs containing the job ID, transition and duration, but do not log names, access tokens or payment data. If personal information is collected, retention and deletion rules should align with the Privacy Act and the organisation’s privacy policy.

Test, monitor and deploy safely

Test state transitions independently from the scheduler. Include cases for successful completion, transient failures, maximum retries, expired jobs, duplicate scheduler execution and a client requesting another user’s job. Integration tests with Testcontainers can verify the actual database locking and timestamp behaviour.

Expose metrics for queue depth, processing duration, failure rate, retry count and polling volume. Alerts can identify a growing backlog before customers in Sydney or Perth notice delays. Run the service in an Australian cloud region when data residency, latency or contractual requirements make that appropriate, and document any cross-border provider involved in processing.

Production recommendations for reliable polling

A polling API should remain useful when workers restart, a provider times out or a client disconnects. Persisted state, bounded retries and transparent status transitions give Spring Boot applications a dependable foundation for reports, authentication workflows and payment-related integrations across the Australian market.