Late Events Arrive After the Window Has Closed

If you publish a daily total the moment the watermark passes the window end, a later row can still fall inside that day by event time. You then either change a number someone has already copied, or you drop the row. Allowed lateness is how long you keep the window state after the end so a late row can still update it. When the watermark passes the end plus that lateness, Flink deletes the state. The row keeps its event time. The published total does not change.

The count in the picture stays at 1,402 when the late order appears. Event time put the order in that day. Processing time only marks when the machine noticed it. The window uses event time. The published total stays 1,402.

Two clocks ride on every row

Apache Flink's page on Timely Stream Processing separates two clocks, and they answer different questions.

Event time is when the event happened on the device that produced it. It is a field inside the row, written before the row reaches Flink. An event-time window uses only this clock. Tuesday's window contains every order whose event time falls inside Tuesday, whatever morning the row happens to be processed.

Processing time is the system clock of the machine running the operator, at the moment the operator sees the row. A processing-time window files the order on the day it arrived. That window is easy. It needs no coordination. Flink's own description is that it also fails determinism. The same order lands in a different hour if the queue is slow, if an operator stalls, or if you replay the topic next week. A daily revenue number that moves because a consumer restarted records when the machines saw the rows. It does not record when the orders happened.

Event time picks the window. Processing time is only when the machine noticed the row.

The job in this piece is an event-time job. Tuesday means the event-time day. That choice is what creates lateness as a concept. Under processing time a row cannot be late for the hour it arrives in, because arrival defines the hour. Under event time a row can belong to a day the operator has already finished. The watermark is how the operator decides that it has finished.

There is a third clock Flink names, ingestion time, which is when the row entered Flink. It behaves like processing time stamped once at the source. It does not come up again here. The collision that matters is the gap between event time and processing time, and the habit of letting the later clock edit a total computed from the earlier one.

When does a window actually close?

A window does not close because the wall clock struck midnight. It closes because event time, as the operator understands it, has moved past the window's end. The tool for that understanding is a watermark.

A watermark carrying time t flows in the stream alongside the records. Flink's definition is a declaration. Event time in that stream has reached t, and there should be no more elements with a timestamp older than or equal to t. Once the watermark reaches an operator, the operator may advance its internal event-time clock to t. A window operator waiting on Tuesday can then see that Tuesday is over and emit the aggregate.

The declaration is a claim the source makes, not a fact the network guarantees. Flink says so in the lateness section of the same page. Elements can show up after a watermark of t with timestamps older than or equal to t. In real pipelines some of those elements can be delayed without any useful bound. You can push the watermark later to wait for more of them, and you pay that wait on every window, including the ones that were already complete. Most jobs refuse that trade. They let the watermark track a chosen bound, often the maximum event time seen so far minus a fixed lag, and they deal with the rows that violate the declaration as a separate case.

Watermarks are born at the sources, one per parallel subtask. Downstream, an operator that reads more than one input does not keep a private clock per input after the merge. Its event time is the minimum of the event times of its inputs. The page states that for unions and for operators that follow a keyBy or a partition. One slow partition therefore holds every key back. A watermark is not "the newest event we have seen on this key." It is "the oldest of the inputs feeding this operator has only reached here." A healthy partition can be full of Wednesday's orders and still be unable to close Tuesday, because another partition has not produced a watermark past Tuesday yet. People debugging a stuck daily job often look at the key that seems late. The clock they should read is the minimum across the inputs.

There is a second way event time moves that is easy to forget during a backfill. Event time does not have to crawl along next to processing time. A job replaying a Kafka topic can run through weeks of event time in a few seconds of processing time, because the data is already buffered. The watermark races forward with the replay. Windows close in a burst. A late-row policy you tested on a live stream, where the watermark lags the wall clock by a minute, will behave differently when a backfill drives the watermark to the end of the file before the side output has been wired. The same function of watermark and window end decides both runs. The speed is what changes.

Set allowed lateness before you publish

By default, a row that arrives after the watermark has passed the end of its window is dropped. Flink's Windows page says the allowed lateness is 0 unless you set it. Zero means the grace line and the window end are the same instant. The first row that shows up after the watermark passes the end never joins the aggregate, and unless you asked for a side output it leaves no trace in the job at all.

Allowed lateness is how long, past the window end, you are still willing to accept a row into that window. The page's own picture is a five-minute tumbling event-time window with one minute of allowed lateness. The window for 12:00 to 12:05 is created when the first element in that interval arrives. It is removed when the watermark passes 12:06. Until that removal, a late element is still added to the window. With the default EventTimeTrigger, adding it can make the window fire again. Those later emissions are late firings. The first emission is the main firing. Flink keeps the window state around specifically so the late firing has something to update. When the watermark passes the end plus the allowed lateness, the state is deleted. A row that arrives after the deletion cannot update a window that is no longer there.

Window end on 30 October, grace ending on 31 October, the watermark on 1 November, and order 8841 on 2 November.

Read the line as the policy. The window end in the picture is 30 October 00:00. Allowed lateness is 24 hours, so Flink removes the state when the watermark passes 31 October 00:00. A watermark at 1 November is past that deletion. An arrival at 2 November 02:14 is not in the "fire again" band. It is past the deletion. Setting allowed lateness to a week would keep Tuesday's state until 6 November, and a row this late would be a late firing instead of a discard. That setting would also keep every other day's state for a week, and it would invite a stream of updated results for numbers you may already have published. You pick the lateness from how wrong a sealed day is allowed to be. A larger number is not safer by itself.

Two bounds get mixed up in reviews, and they should not. The watermark's built-in lag is how late a row can be and still count as on time. Allowed lateness is how late a row can be after the window has closed and still update it. Stretching the watermark lag delays every main firing, including quiet days. Stretching allowed lateness does not delay the main firing. It keeps state, and it produces extra firings. If the complaint is "the dashboard is an hour behind," look at the watermark lag. If the complaint is "Tuesday changed on Thursday," look at allowed lateness and at what the sink does with a late firing.

Three places a late row can land

Take a row whose event time falls inside the window. Half-open, so Tuesday is from 29 October 00:00 inclusive to 30 October 00:00 exclusive. Order 8841 at 18:02 is inside. A row at 30 October 00:00 exactly belongs to Wednesday. From there the watermark picks one of three landings.

While the watermark is still before the window end, the window is open. The row joins the aggregate. The main firing has not happened, or it has not happened for the last time if your trigger fires early. This is the ordinary case, and it is the one the count 1,402 was built from. Those 1,402 orders were present, with event times on Tuesday, before the watermark passed 30 October.

Once the watermark is past the window end, and not yet past the end plus the allowed lateness, the row is late and the window still has state. Flink adds the row and, with EventTimeTrigger, can fire the window again. The docs are direct about what that second emission means. Elements from a late firing are updated results of a previous computation. The stream then contains more than one result for the same window. Your sink has to account for the duplicate or deduplicate it. Ignoring that sentence is how Tuesday's revenue becomes 1,402 in the morning and 1,403 on Thursday, in the same table, with no version a reader can see. The pipeline did what it was configured to do. The publication pretended there was only one result.

Once the watermark has passed the end plus the allowed lateness, the window and its state are gone. The row cannot be added. By default it is dropped. If you called sideOutputLateData with an output tag, the discarded row is available on getSideOutput of that tag, as its own stream. That stream is not a second firing of the window. It does not carry an updated count. It carries the row the window refused. Order 8841, with a watermark of 1 November and a grace line of 31 October, is this third landing.

An open window takes the row. A late firing emits the window again. A side output keeps the row after the state is gone.

A small function draws the same three lines, plus a fourth return for a row that was never in this window. watermark.Before(windowEnd) is "the watermark has not passed the end." watermark.Before(windowEnd + allowed) is "the watermark has not passed the end plus the allowed lateness," which is the 12:06 line in Flink's five-minute example. A watermark equal to the deadline has not passed it, so the row is still a late firing. The next watermark, any time after the deadline, is the side output.

func Landing(event, windowEnd, watermark time.Time, allowed time.Duration) string {
	if !event.Before(windowEnd) {
		return "other-window"
	}
	deadline := windowEnd.Add(allowed)
	switch {
	case watermark.Before(windowEnd):
		return "open"
	case watermark.Before(deadline):
		return "late-firing"
	default:
		return "side-output"
	}
}

Call it with order 8841. event is 29 October 18:02, windowEnd is 30 October 00:00, allowed is 24 hours, watermark is 1 November 00:00. The event is before the window end, so it is not other-window. The deadline is 31 October 00:00. The watermark is after the deadline. The function returns side-output. Change the watermark to 30 October 12:00 and it returns late-firing. Change it to 29 October 20:00 and it returns open. The three answers are the whole policy. What you do with each answer is the sink's problem, and Flink will not pick it for you. A late firing is another result on the main stream. A side output is a different stream. Dropping the row is what you get if you configure neither a reaction to the second result nor a side output.

The function does not know whether anyone has already read 1,402. That fact lives outside the window operator, in whatever table or file or dashboard the main firing was written to. Classifying the row is the operator's job. Deciding that a published 1,402 must not be edited in place is a publication rule sitting on top of the classification. Mixing them up produces two opposite bugs. One bug writes every late firing over the published total and calls the table "corrected." The other bug drops late firings on the floor because someone heard that late data is dangerous, and Tuesday stays wrong by an amount the job had in memory and chose not to emit.

Who already copied Tuesday?

The main firing is the moment the number leaves the job. After that, copies exist that the job does not control.

A finance export ran at 06:00 on 30 October and stored 1,402 in a sheet a person mailed. A downstream feature job read the daily table and trained on it overnight. An on-call dashboard pinned the number in a screenshot. None of these listeners are subscribed to a retraction. If the pipeline emits a late firing and the sink runs UPDATE daily SET total = 1403 WHERE day = '2026-10-29', the table now disagrees with every copy that left. The table is closer to the event-time truth. The copies are what people acted on. You cannot tell, from the table alone, that the truth moved.

Flink already told you the stream can hold multiple results for one window. The honest sink stores them as multiple results. The main firing is a row with the window key, the total, and a version of 1, marked sealed once you are willing to let readers treat it as the number for that day. A late firing is a new row with the same window key, the new total, and a version of 2. It does not replace version 1. A reader that knows how to fold versions can compute the latest total. A reader that copied version 1 still has a true statement about what the pipeline had published, and a place to look up that a later version exists.

This is a different problem from a retry. A retried request is the same intent delivered twice, and the fix is to recognize it and do the work once. Order 8841 is not a second delivery of an order already in the 1,402. It is a first delivery of an order whose event time is old. Deduplicating on order_id inside the window is still worth doing, because a broker can redeliver. It does not answer lateness. The row is new. The window is what is old.

The cost of versions is storage and a rule readers must follow. You keep more than one total per day. You have to say, in the table's grain, that the sealed version is the one that shipped and later versions are corrections. The cost of an in-place update is worse and harder to see. Every consumer that read the old total is now silently wrong, and the only audit trail is a database log most of them will never open. I would pay for the versions on any daily number a person or another job will copy. I would not pay for them on a window that is still open and has never fired, because nothing has been copied yet and the aggregate in state is the only copy.

Keep the refused row

The default, allowed lateness 0 and no side output, is a quiet drop. The row arrives, the watermark is past the window end, the operator discards the element, and the metric you meant to watch is whatever you added yourself. If you added nothing, the drop is invisible. Tuesday stays 1,402. The order exists in the source topic. The daily table does not mention it. A reconciliation that compares source counts to published counts will eventually scream, often in a month-end job that was not built to name the missing order_id.

sideOutputLateData is the trace. The tag is an OutputTag you pass to the windowed stream. The late stream you get back is ordinary records, the ones the window refused, and you sink them somewhere that is not the sealed total. A table of refused rows, keyed by the order and by the window they would have joined, is enough. From that table you can count how late you are, in rows and in money, without ever adding that money into a day you already sealed. You can also see whether the grace is too tight. A side output that is a large fraction of the day's total means the watermark is ahead of the data, and the fix is the watermark lag or the allowed lateness, not a manual edit of old days.

Do not sink the side output back into the sealed total on a timer. That is an in-place update with extra steps. The side output is the evidence that the row missed the window. The correction, if you publish one, is a new version, produced by a person or a job that meant to publish a correction. An automatic fold of the side output into yesterday's number recreates the late firing you just refused, and it does it outside the window operator, where the allowed-lateness budget no longer applies.

Session windows add one more surprise the tumbling day does not have. A late element can bridge two sessions and merge windows that had already fired. The same docs that describe late firings warn about this. A daily tumbling window cannot merge with its neighbor, because the bounds are fixed. If your "day" is actually a session gap, a late row is not only a missed count. It can change which events were considered the same session. The side output still matters, and the sealed session result needs versions even more, because the correction may be a different grouping rather than a total plus one.

The sealed total stays 1,402

Publish the main firing as version 1 and mark it sealed. Leave it. When a late firing arrives inside the grace, write version 2 beside it, with the same window key and the new total, and point the "latest" reader at the greatest version. When a row arrives past the grace, write the row to the side-output table and do not mint a version that changes 1,402. Anyone who needs the event-time truth after the fact joins the sealed total to the side output and accepts that they are computing a different number from the one that shipped.

Order 8841 takes that last path. Its event time is Tuesday. The operator sees it the following Monday. The watermark has already passed the grace line. The function returns side-output. The daily table keeps 1,402 as version 1. The refused table names the order, the window end it missed, and the watermark that refused it. A person reconciling the month can add 1 when they build a restated total. They cannot claim the shipped number was 1,403. The shipped number was 1,402.