Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mqtt gateway refactoring due to OOM #10804

Merged
merged 6 commits into from
May 22, 2024
Merged

mqtt gateway refactoring due to OOM #10804

merged 6 commits into from
May 22, 2024

Conversation

YevhenBondarenko
Copy link
Contributor

Pull Request description

MQTT transport OOM - AbstractGatewaySessionHandler
https://thingsboard-portal.atlassian.net/browse/PROD-3598

image

General checklist

  • You have reviewed the guidelines document.
  • Labels that classify your pull request have been added.
  • The milestone is specified and corresponds to fix version.
  • Description references specific issue.
  • Description contains human-readable scope of changes.
  • Description contains brief notes about what needs to be added to the documentation.
  • No merge conflicts, commented blocks of code, code formatting issues.
  • Changes are backward compatible or upgrade script is provided.
  • Similar PR is opened for PE version to simplify merge. Crosslinks between PRs added. Required for internal contributors only.

Front-End feature checklist

  • Screenshots with affected component(s) are added. The best option is to provide 2 screens: before and after changes;
  • If you change the widget or other API, ensure it is backward-compatible or upgrade script is present.
  • Ensure new API is documented here

Back-End feature checklist

  • Added corresponding unit and/or integration test(s). Provide written explanation in the PR description if you have failed to add tests.
  • If new dependency was added: the dependency tree is checked for conflicts.
  • If new service was added: the service is marked with corresponding @TbCoreComponent, @TbRuleEngineComponent, @TbTransportComponent, etc.
  • If new REST API was added: the RestClient.java was updated, issue for Python REST client is created.
  • If new yml property was added: make sure a description is added (above or near the property).

@YevhenBondarenko YevhenBondarenko added this to the 3.7 milestone May 16, 2024
smatvienko-tb
smatvienko-tb requested changes May 16, 2024
View reviewed changes
Copy link
Contributor

@smatvienko-tb smatvienko-tb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, the code becomes much more readable with less overhead. A few points to work on:

  1. check device - single method, single logic
  2. How does the client will get an error if some conversion goes wrong? Currently it
    swallowed.
  3. Make sure that the netty workers does not occupied for a long time . Otherwise TCP clients will fell off

...c/main/java/org/thingsboard/server/transport/mqtt/session/AbstractGatewaySessionHandler.java Outdated
}

String deviceName = deviceEntry.getKey();
T deviceCtx = devices.get(deviceName);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should use a single method
checkDeviceConnected(deviceName)

That returns future (immediate or real future)

Then check if the future is completed and successful - process now or add a callback

Having a custom device check (accessing internal map) looks like we have duplicated the logic

...c/main/java/org/thingsboard/server/transport/mqtt/session/AbstractGatewaySessionHandler.java Outdated

deviceMsgList.forEach(telemetryMsg -> {
String deviceName = checkDeviceName(telemetryMsg.getDeviceName());
T deviceCtx = devices.get(deviceName);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the same as above with checkDeviceConnected

...c/main/java/org/thingsboard/server/transport/mqtt/session/AbstractGatewaySessionHandler.java Outdated
if (deviceCtx != null) {
processClaimDeviceMsg(deviceCtx, deviceEntry.getValue(), deviceName, msgId);
} else {
Futures.addCallback(checkDeviceConnected(deviceName), new FutureCallback<>() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

checkDeviceConnected and onDeviceConnect(deviceName, DEFAULT_DEVICE_TYPE) are doing the same checks and wraps
we can remove one of the method

the onDeviceConnect do all the job
image

...c/main/java/org/thingsboard/server/transport/mqtt/session/AbstractGatewaySessionHandler.java Outdated
if (deviceCtx != null) {
processClaimDeviceMsg(deviceCtx, claimDeviceMsg.getClaimRequest(), deviceName, msgId);
} else {
Futures.addCallback(checkDeviceConnected(deviceName), new FutureCallback<>() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can use the util DonAsynchron to simplify code

...c/main/java/org/thingsboard/server/transport/mqtt/session/AbstractGatewaySessionHandler.java Outdated
continue;
}
String deviceName = deviceEntry.getKey();
Futures.addCallback(checkDeviceConnected(deviceName), new FutureCallback<>() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here there is no immediate logic implemented. please, check the future result . it might be already done

...c/main/java/org/thingsboard/server/transport/mqtt/session/AbstractGatewaySessionHandler.java Outdated
TransportProtos.PostAttributeMsg postAttributeMsg = JsonConverter.convertToAttributesProto(msg.getAsJsonObject());
transportService.process(deviceCtx.getSessionInfo(), postAttributeMsg, getPubAckCallback(channel, deviceName, msgId, postAttributeMsg));
} catch (Throwable e) {
log.warn("[{}][{}][{}] Failed to process device attributes command: [{}]", gateway.getTenantId(), gateway.getDeviceId(), deviceName, msg, e);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How will the client on the remote end get an error if some conversion does not go well? Logs are for the ThingsBoard sysadmin purposes only.

@ashvayka ashvayka merged commit 3ab5a67 into thingsboard:master May 22, 2024
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@smatvienko-tb smatvienko-tb smatvienko-tb requested changes

@ViacheslavKlimov ViacheslavKlimov Awaiting requested review from ViacheslavKlimov

Assignees
No one assigned
Labels
bug enhancement
Projects
None yet
Milestone
3.7
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@YevhenBondarenko @smatvienko-tb @ashvayka