WebSocket ping timeout，KernelRestarter，help!!!

[1201wen]

I need to train the model, but when I am loading the data, the following error occurs frequently. Is there a good way to solve it? Please continue to help me. Thank you.

[W 15:53:09.718 NotebookApp] WebSocket ping timeout after 91528 ms.
[I 15:53:36.375 NotebookApp] Starting buffering for 4d0c3291-f90b-4f19-ad85-103b8a5d3b3b:043a694c25924b7092ba09d397cf9f00
[I 15:55:34.485 NotebookApp] Restoring connection for 4d0c3291-f90b-4f19-ad85-103b8a5d3b3b:043a694c25924b7092ba09d397cf9f00
[W 16:11:05.389 NotebookApp] WebSocket ping timeout after 91883 ms.
[I 16:11:35.863 NotebookApp] Starting buffering for 4d0c3291-f90b-4f19-ad85-103b8a5d3b3b:043a694c25924b7092ba09d397cf9f00
[I 16:14:40.473 NotebookApp] Restoring connection for 4d0c3291-f90b-4f19-ad85-103b8a5d3b3b:043a694c25924b7092ba09d397cf9f00
[W 16:27:34.165 NotebookApp] WebSocket ping timeout after 98880 ms.
[I 16:27:53.175 NotebookApp] Starting buffering for e11abe95-c9ca-4e8a-bc4c-84b2e3b1c0d1:d7fac0aaa4264d9d867d1319a7d50e2e
[I 16:32:44.998 NotebookApp] Restoring connection for e11abe95-c9ca-4e8a-bc4c-84b2e3b1c0d1:d7fac0aaa4264d9d867d1319a7d50e2e
[I 16:34:56.914 NotebookApp] Starting buffering for e11abe95-c9ca-4e8a-bc4c-84b2e3b1c0d1:d7fac0aaa4264d9d867d1319a7d50e2e
[I 16:35:45.108 NotebookApp] Restoring connection for e11abe95-c9ca-4e8a-bc4c-84b2e3b1c0d1:d7fac0aaa4264d9d867d1319a7d50e2e
[W 16:37:47.290 NotebookApp] WebSocket ping timeout after 124862 ms.
[E 16:37:51.355 NotebookApp] Uncaught exception GET /api/kernels/e11abe95-c9ca-4e8a-bc4c-84b2e3b1c0d1/channels?session_id=d7fac0aaa4264d9d867d1319a7d50e2e (127.0.0.1)
    HTTPServerRequest(protocol='http', host='127.0.0.1:8889', method='GET', uri='/api/kernels/e11abe95-c9ca-4e8a-bc4c-84b2e3b1c0d1/channels?session_id=d7fac0aaa4264d9d867d1319a7d50e2e', version='HTTP/1.1', remote_ip='127.0.0.1')
    Traceback (most recent call last):
      File "/home/wen/anaconda3/lib/python3.9/site-packages/tornado/websocket.py", line 956, in _accept_connection
        await open_result
      File "/home/wen/anaconda3/lib/python3.9/asyncio/tasks.py", line 328, in __wakeup
        future.result()
    tornado.util.TimeoutError: Timeout
[W 16:38:34.177 NotebookApp] Replacing stale connection: e11abe95-c9ca-4e8a-bc4c-84b2e3b1c0d1:d7fac0aaa4264d9d867d1319a7d50e2e
[I 16:42:48.829 NotebookApp] Starting buffering for e11abe95-c9ca-4e8a-bc4c-84b2e3b1c0d1:d7fac0aaa4264d9d867d1319a7d50e2e
[W 16:43:03.934 NotebookApp] Replacing stale connection: e11abe95-c9ca-4e8a-bc4c-84b2e3b1c0d1:d7fac0aaa4264d9d867d1319a7d50e2e
[W 16:46:18.039 NotebookApp] Replacing stale connection: e11abe95-c9ca-4e8a-bc4c-84b2e3b1c0d1:d7fac0aaa4264d9d867d1319a7d50e2e
[W 16:51:03.918 NotebookApp] Replacing stale connection: e11abe95-c9ca-4e8a-bc4c-84b2e3b1c0d1:d7fac0aaa4264d9d867d1319a7d50e2e
[W 16:56:12.279 NotebookApp] Replacing stale connection: e11abe95-c9ca-4e8a-bc4c-84b2e3b1c0d1:d7fac0aaa4264d9d867d1319a7d50e2e
[W 16:59:21.146 NotebookApp] WebSocket ping timeout after 95563 ms.
[I 16:59:40.471 NotebookApp] Starting buffering for 4d0c3291-f90b-4f19-ad85-103b8a5d3b3b:043a694c25924b7092ba09d397cf9f00
[W 17:03:28.672 NotebookApp] Replacing stale connection: e11abe95-c9ca-4e8a-bc4c-84b2e3b1c0d1:d7fac0aaa4264d9d867d1319a7d50e2e
[I 17:11:30.910 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports

[RRosio]

Hi @1201wen, can you share some additional information, like a code snippet that can help us reproduce this issue? Are you attempting to load a large amount of data? Also, could you share the version of Notebook you are using? Would you be able to try the latest Notebook version 7.1?

[1201wen]

Hi@RRosio ,thank you for your reply.The following is my relevant code when loading data. When I use this code for processing, the above problem will occur. When I change parallel processing to serial processing, the problem can be solved in the data loading phase. However, when entering the model training phase, the problem will occur again. I do need to load a lot of data, and my dataset folder cannot be opened properly through jupyter. The version I use is 6.4.12. Is there any improvement between this version and the latest version in dealing with this problem?

def loadAstData(self):
        self.ast_data = Parallel(n_jobs=2, backend='multiprocessing')(
            delayed(getNpy)(os.path.join('./dataset', src[0].replace('/', '.') + '.json'), self.max_node)
            for src in self.paths
        )

def getNpy(path, max_node):
    try:
        with open(path, 'r') as file_object:
            data = json.load(file_object)
        data[3] = [tokenizer.convert_tokens_to_ids(clean_str(item)[0: max_node]) for item in data[3]]
        for index, src in enumerate(data[3]):
            if len(src) == 0:
                data[3][index].append(1)
        return data
    except Exception as e:
        print(f"加载{path}时错误：{str(e)}")
        return None

[RRosio]

Hi @1201wen thank you for providing the additional information. If you can try out using the resource manager plugin, https://github.com/jupyter-server/jupyter-resource-usage?tab=readme-ov-file#jupyterlab-3x-and-notebook-6x, (thanks to @krassowski) to monitor your notebook's memory usage. If your memory issue is normal, please let us know. The reason we suggested Notebook 7 was that it might have more descriptive error messaging.

[RRosio]

Hi @1201wen, just following up here, were you able to try the jupyter-resource-usage plugin?

[1201wen]

Hi@RRosio, I'm sorry that I have some other things going on recently and have temporarily put the model training on hold, so I can't tell you the new progress. Please give me an email if it's convenient for you. I'll let you know if there are any new progress in the future.

[1201wen]

@RRosio

[RRosio]

Thank you the update @1201wen! Lets keep this discussion public on this repository. As we cannot verify that this is an issue with the Notebook application itself, I will close this issue but please feel free to reopen and update this once you have had the chance to look over your resource usage with the suggested plugin.