Skip to content

Log the detailed exception when reconciliation #935

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

zuston
Copy link
Contributor

@zuston zuston commented Feb 15, 2022

When implementing the controller like TomcatReconciler and throw exception when accepting k8s event, the log will not throw the exception stacktrace. Due to this limitation, I spent lot of time to dig operator framework and found the fundamental failure.

So i hope this PR to solve this problem.

@zuston
Copy link
Contributor Author

zuston commented Feb 15, 2022

@csviri @metacosm Please review it.

Copy link
Collaborator

@csviri csviri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@csviri csviri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually this would be better to catch in ControllerExecution. @zuston could you change it pls ? or I can issue a new PR

Copy link
Collaborator

@metacosm metacosm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@csviri
Copy link
Collaborator

csviri commented Feb 15, 2022

Or rather here:

  public PostExecutionControl<R> handleExecution(ExecutionScope<R> executionScope) {
    try {
      return handleDispatch(executionScope);
    } catch (KubernetesClientException e) {
      log.info(
          "Kubernetes exception {} {} during event processing, {} failed",
          e.getCode(),
          e.getMessage(),
          executionScope);
      return PostExecutionControl.exceptionDuringExecution(e);
    } catch (RuntimeException e) {
      log.error("Error during event processing {} failed.", executionScope, e);
      return PostExecutionControl.exceptionDuringExecution(e);
    }
  }

Wondering why this does not work.

@metacosm
Copy link
Collaborator

actually this would be better to catch in ControllerExecution. @zuston could you change it pls ? or I can issue a new PR

I was wondering about that, indeed, and thought that we could do that in a subsequent PR.

@csviri
Copy link
Collaborator

csviri commented Feb 15, 2022

Probably just this should be an error level:

   } catch (KubernetesClientException e) {
      log.info(
          "Kubernetes exception {} {} during event processing, {} failed",
          e.getCode(),
          e.getMessage(),
          executionScope);
      return PostExecutionControl.exceptionDuringExecution(e);

Or just removed and handle all the runtime exceptions as it is now without this part.

@zuston
Copy link
Contributor Author

zuston commented Feb 15, 2022

Thanks for you quick reply @csviri @metacosm .
Maybe this also should be added in cleanup and other methods exposed for users.

@csviri
Copy link
Collaborator

csviri commented Feb 15, 2022

Thanks for you quick reply @csviri @metacosm . Maybe this also should be added in cleanup and other methods exposed for users.

The thing is, for me it seems that the only problem is we make different logging for client exception, as you can see we catch all the exceptions also for cleanup in the quoted code. So I would just remove that special case for the client.

@csviri
Copy link
Collaborator

csviri commented Feb 15, 2022

created this PR: #936
would rather do it this way and just close this.

But thx @zuston to pointing this out!

@zuston
Copy link
Contributor Author

zuston commented Feb 15, 2022

created this PR: #936 would rather do it this way and just close this.

But thx @zuston to pointing this out!

I like this general fix. Thanks @csviri.

Could you let me join into slack group, now I use this project to develop our elastic-yarn operator.

@csviri
Copy link
Collaborator

csviri commented Feb 15, 2022

Could you let me join into slack group, now I use this project to develop our elastic-yarn operator.

We don't have a slack group, there is a Discord server, use this to join: https://discord.gg/DacEhAy
Or join the generic Kuberentes slack, usually watching this channel: https://kubernetes.slack.com/archives/CAW0GV7A5

@csviri csviri closed this Feb 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants