GKEStartJobOperator (deferrable=True) does not show container logs on failure

[atulgandhi1137]

Question:
I’m using GKEStartJobOperator to run jobs on GKE and noticed a difference in logging behavior depending on whether deferrable is enabled.

What I’m seeing:

• With deferrable=True:

      On success → container logs are visible in Airflow task logs
      On failure (e.g., BackOffLimitExceeded) → only job-level error is shown, container logs are missing 
  

• With deferrable=False:

      On failure → container logs are correctly shown in task logs 
  

My questions:

• Is this expected behavior when using deferrable=True?
• Or should container logs also be available on failure in deferrable mode?

Why this matters:
When jobs fail, not having container logs makes debugging difficult.

If this is expected, is there a recommended workaround to fetch container logs in deferrable mode?

[cookesan]

From the current provider source, your observation looks like a real deferrable-path difference.

In the non-deferrable path, KubernetesJobOperator waits for the job, fetches requested container logs when get_logs is enabled, and only then checks is_job_failed().

In the deferrable path, the trigger waits for the job and returns an event with status = "error" when the job failed. Then execute_complete() immediately raises on that error status before it reaches the later if self.get_logs: block that would fetch pod logs.

So I would not treat this as user error. The practical workarounds are to run this operator with deferrable=False when failure logs are important, or fetch the pod logs separately in an Airflow failure callback/downstream diagnostic task. The upstream fix would be to retrieve the requested logs in execute_complete() before raising the job-failed exception.