Insights into Company
← Spring Boot

Kafka & Event-Driven Architecture

01 What is Kafka and why use it with Spring Boot?

Solution and explanation

Solution:

Suppose a candidate applies for a job.

The Application Service performs the core database transaction.

After the application is successfully created, several things may need to happen:

  1. Send email.
  2. Notify recruiter.
  3. Update analytics.
  4. Create audit record.
  5. Update recommendation data.
  6. Update search-related statistics.

The application API should not necessarily wait for all of these operations.

An event can be published:

ApplicationSubmitted

Consumers can independently process it.

Example:

Application Service

v Kafka

+---- Notification Service

+---- Analytics Service

+---- Audit Service

+---- Recommendation Service

If Notification Service temporarily fails, Kafka can retain the event until the consumer can process it according to the configured retention/processing strategy.

Each consumer should be idempotent.

Run this example 
        @Service
        public class ApplicationEventPublisher {

            private final KafkaTemplate<String, ApplicationSubmittedEvent> kafka;

            public ApplicationEventPublisher(
                KafkaTemplate<String, ApplicationSubmittedEvent> kafka
            ) {
                this.kafka = kafka;
            }

            public void publish(
                ApplicationSubmittedEvent event
            ) {

                kafka.send(
                    "application-submitted",
                    event.applicationId().toString(),
                    event
                );
            }
        }

        @KafkaListener(
            topics = "application-submitted",
            groupId = "notification-service"
        )
        public void consume(
            ApplicationSubmittedEvent event
        ) {

            notificationService
                .sendApplicationConfirmation(event);
        }
Hands-on challenge

Create ApplicationSubmitted and ApplicationStatusChanged events and process them with separate consumer groups.

Quick check

Which statement is correct?

02 What is a Kafka partition and consumer group?

Solution and explanation

  • Solution:
  • Suppose:
  • application-submitted
  • has 3 partitions.
  • Partition 0
  • Partition 1
  • Partition 2
  • A consumer group contains:
  • Consumer A
  • Consumer B
  • Consumer C
  • The partitions can be distributed among consumers.
  • The important rule is:
  • One partition is processed by only one consumer within the same consumer group at a time.
  • If another service uses another consumer group:
  • notification-service
  • analytics-service
  • both groups receive their own processing view of the topic.
  • This is useful for a job portal.
  • The same ApplicationSubmitted event may need to be consumed by:
  • Notification Service
  • Analytics Service
  • Audit Service
  • They should not be forced into one consumer group because each represents an independent business function.
  • Partition keys also matter.
  • If applicationId is the key, events for the same application can be routed to the same partition, preserving order within that partition.
Run this example 
        spring:
          kafka:
            consumer:
              group-id: notification-service

            producer:
              key-serializer: org.apache.kafka.common.serialization.StringSerializer
              value-serializer: org.springframework.kafka.support.serializer.JsonSerializer

            consumer:
              key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
              value-deserializer: org.springframework.kafka.support.serializer.JsonDeserializer
Hands-on challenge

Create separate consumer groups for notification, analytics and audit processing.

Quick check

Which statement is correct?

03 How do you make Kafka consumers idempotent?

Solution and explanation

Solution:

Suppose Kafka delivers:

eventId = EVT-1001

The Notification Service processes it and sends an email.

Now the same event is delivered again.

If the service blindly processes it:

Email 1 Email 2

The candidate receives duplicate notifications.

The consumer should maintain a processed-events table.

Example:

processed_events

id event_id consumer_name processed_at

Before processing:

SELECT event_id FROM processed_events WHERE event_id = 'EVT-1001' AND consumer_name = 'notification-service';

If it exists:

Do not process again.

If it does not exist:

  1. Process event.
  2. Record event ID.
  3. Commit transaction.

A unique constraint should protect against concurrent duplicate processing.

Idempotency is essential because distributed messaging systems can involve redelivery.

Run this example 
        CREATE TABLE processed_events (
            id BIGINT PRIMARY KEY AUTO_INCREMENT,
            event_id VARCHAR(100) NOT NULL,
            consumer_name VARCHAR(100) NOT NULL,
            processed_at TIMESTAMP NOT NULL,
            UNIQUE(event_id, consumer_name)
        );

        @KafkaListener(
            topics = "application-submitted",
            groupId = "notification-service"
        )
        @Transactional
        public void consume(ApplicationSubmittedEvent event) {

            if (processedEventRepository
                    .existsByEventIdAndConsumerName(
                        event.eventId(),
                        "notification-service"
                    )) {

                return;
            }

            notificationService.send(event);

            processedEventRepository.save(
                new ProcessedEvent(
                    event.eventId(),
                    "notification-service"
                )
            );
        }
Hands-on challenge

Send the same Kafka event three times and verify that the business operation occurs only once.

Quick check

Which statement is correct?

04 What is Spring Batch versus Kafka?

Solution and explanation

  • Solution:
  • Spring Batch and Kafka solve different problems.
  • Spring Batch:
  • Good for controlled large-scale processing.
  • Example:
  • Every night:
  • Process 5 million employee records.
  • Read -> process -> write in chunks.
  • Kafka:
  • Good for event streaming and asynchronous communication.
  • Example:
  • Candidate submits application.
  • ApplicationSubmitted event is consumed by:
  • Notification
  • Analytics
  • Audit
  • Recommendation
  • A system can use both.
  • Example:
  • Kafka receives:
  • EmployeeSalaryApproved
  • A Spring Batch process can later process a large reconciliation file.
  • Do not select a technology merely because the data volume is large.
  • The workflow determines the appropriate tool.
Run this example 
        Kafka:

        ApplicationSubmitted
                |
                +--> Notification
                +--> Analytics
                +--> Audit

        Spring Batch:

        Payroll File
              |
        ItemReader
              |
        ItemProcessor
              |
        ItemWriter
              |
        Payroll Database
Hands-on challenge

Identify which parts of a job portal should use Kafka and which should use Spring Batch.

Quick check

Which statement is correct?