After updating to 1.70 my backup job with the internal app is not working anymore. I start the backup routine manually. The source data are on the ZimaOS NAS (media data, big files), the target is an TrueNAS Backup server. In the folder are several hundred files. After the backup routine did not work (after starting the was no traffic of any kind between the servers), i deleted the backup routine and created a new one from scratch. Same result - nothing is happening. Any advice appreciated. In 1.62 everything worked like a charm.
Addition to my post:
System:
ZimaOS 1.7.0 (final)
Hardware: Ryzen 7 PRO 8845HS, 64 GiB RAM
Previously working fine on 1.6.2
Problem:
The icewhale-files-backup process grows to 40+ GiB RSS within a few minutes, pushing the system close to OOM (RAM usage climbing from a ~11% baseline to 92%). The growth repeats after restarting the process.
Log excerpt:
error failed to get device model {"error": "failed to get device model: Get \"http://192.168.1.61:9527/\": dial tcp 192.168.1.61:9527: connect: connection refused", "func": "service.GetRemoteStorages", "file": "/home/runner/work/IceWhale-Files-Backup/IceWhale-Files-Backup/service/storage.go", "line": 349}
Reproduction / Observations:
The backup target is a TrueNAS SCALE system on the local network (online and reachable via SMB/NFS), but without a CasaOS/ZimaOS API on port 9527. Every attempt by GetRemoteStorages to fetch this target’s device model fails with connection refused.
These errors repeat every few minutes up to roughly every 30 minutes, with the process’s RSS increasing each time — looks like a leak per failed retry (an unclosed HTTP client/response body would be a typical cause in Go).
Multiple distinct PIDs of the same service appear in the log (e.g. 1988, 192896, 158522) — suggesting repeated crash-restarts triggered by the OOM killer/watchdog.
Possible additional factor: Whether the backup target is already online/reachable at the start of the backup job, vs. the connection dropping or being established with delay during the job, may play a role — in my case the target system was online throughout and the problem still occurred, but I have not systematically tested the timing (target online before vs. during backup start).
Possible additional factor: The affected backup jobs mostly involved large media files (typically >15 GB). Under 1.6.2, the same backup jobs with the same file sizes and the same target ran without issues. I can’t say for certain whether file size affects the leak rate or is only a coincidental correlation (e.g. because larger files mean longer connection time and thus more retry cycles).
Expected behavior:
As under 1.6.2: on an unreachable remote storage target, the service should log the error and proceed to the next retry cycle without a resource leak, instead of accumulating RAM.
Same problem. Since upgrading to 1.70 the Jellyfin media folder stopped being backed up. Memory shoots to the max and then goes back to normal. Backup loads previous files and crashes to zero.
I can confirm that the 1.7.1 alpha 1 is solving the backup issue. I launched a bunch of big backup tasks and the process icewhale-files-backup only use 118M of RAM.
I would also Like to test 1.71. Alpha. But for me, it’s not that important. I’m pretty new to selfhosting.
Also I would like to say, that the back up is buggy. It kinda works. My ram is also maxing out, but a few smaller folders could be easily backed up.
So at least 2/3 attempts, and the third still running, I can imagine, that it’s a bug in the UI or the conversion from the backend to the front end. The ram maxing is weird, to before. Even though, it also uses some CPU. So I wouldn’t really think, that it’s not working. And the CPU isn’t maxing out.
Also in the backup UI it states, that there are no files, but after manually starting the backup, end when it finishes, it showed the numbers and the sizes. So I think, that that the counter for the files, runs at the same loop, as the backup write process, but it only gets given back, when the loop finished.
Still, even though as it seems, that it works, I would like to test the 1.7.1 alpha, but please also an instruction how to turn that version back, if I fail while upgrading
And a suggestion from myself. Probably an option for ram-limiting would be nice. I currently have 64gb ram, but it uses 35 GB. It’s not that it bothers me, but what happens, if I use an app, like an LLM? Than I have to apps fighting for ram
Hello,
I finally took the plunge and installed version 1.7.1-alpha1.
I first stopped all apps and then started the update.
The update went smoothly.
System came up with normal RAM usage.
I had to retry each backup job once with the “Retry” command, after which everything was OK.
After the first restart, none of my apps started automatically(maybe because I stopped them before).
I had to edit the settings of each app, once and then clicked save, after that all the apps started again.
It looks like the system is running stable, now.
Thx and Kr
Holger
Unfortunately I have to manually start the Backup after any restart, now.
After a few restarts the apps do not start up anymore → rollback to 1.6.2 again, all fixed
