XCP NG Forum

@afk said: Hi everyone, Some of you may already be aware of this but I learned the news this morning. https://www.virtualizationhowto.com/2026/09/leaving-vmware-just-got-harder-after-broadcom-pulled-vddk-downloads/ Essentially, Broadcom has decided, without any announcement, to remove all download links (and even some documentation it seems) for VDDK. This impacts all migration tools that use the VDDK for copying data of VM disks. If you have any version of VDDK, please keep it somewhere safe and make backups. I only have a x86-64 linux release from september 2025 that I used for testing V2V. Obviously, I can't share any link here but the release has the following sha256 hash: 79d215198f1b8fd1d240a16b27ac2543c521ae2afdc1876444c2acf8945d74ca VMware-vix-disklib-9.0.0.0.24742305.x86_64.tar.gz Apparently, web archival links are also taken down. @olivierlambert With Broadcom removing public access to VDDK downloads, we now have users who either cannot obtain VDDK at all or whose existing VDDK installations (main + backup) have become corrupted. This creates a real need for a fully open‑source fallback path inside V2V, one that still meets the performance requirements normally associated with VDDK. The open‑source community has already demonstrated that VDDK‑level throughput is achievable without VMware’s proprietary stack. Several projects have shown that: • Parallel NBD can saturate modern storage bandwidth when implemented correctly • Highly multithreaded block pipelines can out

v0.6.0 — log collection is in. This was the piece the first post said was next, and it works end to end now. A new Collect page runs a per-host collection as a background job: it pulls logs.tgz and the XAPI audit trail from Xen Orchestra, keeps the raw copies, and produces a redacted copy of each. On my own XCP-ng 8.3 pool that came to 434 MiB for the bundle and 769 MiB for the audit trail, 2.3 GiB stored across the five files, with 2,542,805 values masked — almost all of them session tokens. The download is streamed a megabyte at a time and never held in memory, and every chunk is a cancellation point, so Cancel actually stops a transfer rather than waiting it out. A failed or cancelled download deletes its partial file instead of leaving a half-written bundle sitting there looking like a collection. Both copies are kept and marked redacted — safe to send or raw — unmasked. Redaction is lossy, and the raw copy is the only thing that can answer "what did that used to say?" afterwards. Each collection writes the same redaction report the preview page produces, so you can see which rules fired — and which were switched off — before anything leaves the box. In the screenshot I had IPv4, MAC, UUIDs and hostnames turned off, and they show as off rather than as zero hits. That is deliberate: a partly-masked bundle should be obvious before you attach it to a ticket. There is also a retention policy. The newest n collections are kept whatever their age, and only what is left is judge

@florent said: @jr-m4 you can export the heap memory of the nodeJS process by doing kill -SIGUSER2 onte that this will increase a lot the memory consumed by the xo process even when its done exporting the memory . This will help us know what xo is doing at the moment Do you have somewhere I can upload the heapsnapshot? (148MB) Ping @poddingue as well, for visibility

Doing some more testing it doesn't seem to matter how many pools i have in the metadata backup job, it will hang and time out after 20 mins. I am also seeing this on my home install with XO from sources after updating to the latest commit.

On one machine in my lab, /etc/rsyslog.d/xenserver.conf defines 17 outchannels, every one of them at 104857600 bytes, and /var/log/messages isn't one of them; it's written by the stock *.info;mail.none;authpriv.none;cron.none line in /etc/rsyslog.conf and rotated by /etc/logrotate.d/syslog, so nightly and with no size trigger. The docs at https://docs.xcp-ng.org/guides/logs#rsyslog describe the 100 MiB rule but don't list which files it actually covers, which is probably why this is hard to work out without reading the config the way you did. Whether messages can realistically outrun a nightly rotation I don't know, and I'd be guessing if I said either way; the one thing in your favour is that /var/log is its own filesystem, 3.9 GB on the box I looked at (small lab machine, I haven't searched in the code, so I don't know where that size is decided), so it can't take the rest of dom0 with it. Whether that omission is deliberate or just inherited (or even something fancier) is really a question for whoever owns that file, so it might be worth a mention to @Team-OS-Platform-Release.

@poddingue Hey, it's just fun building stuff, even if it ends up in the bit-bucket (as unfotunately that's the destiny of all things anyway). Happy Day

If you are running XOA, not XO from sources, then we could take a look remotely using the support tunnel. Otherwise, make sure that you have patched and rebooted each pool member.

@ olivierlambert said : Follow-up on the questions left open in this thread. Three things came out differently from what I said above, so corrections first, and my earlier headline numbers need revising upward. My earlier ratios were too low: the rig was the bottleneck Everything I posted before ran NVMe to NVMe, where the destination disk saturates around 250 to 300 MB/s and degrades as it fills. That capped the faster arms, arm C worst of all. Rebuilt with RAM-backed SRs on both hosts, no storage ceiling, full four-arm matrix on one rig, 3 runs per arm, interleaved. arm build MiB/s range vs stock stalls B/seg A stock control 71.3 71.2-71.4 1.00x 78.3% 1441.6 B TCP_NODELAY only +socket 303.8 303.1-305.0 4.26x 1.5% 1426.6 C NODELAY + pipelining both 526.6 522.8-528.5 7.39x 1.1% 1425.7 D pipelining only +pipeline 528.3 523.0-531.9 7.41x 16.1% 1442.0 Per-arm spread is 0.3% to 1.7%. The full stack is worth 7.4x, not 5.51x , and TCP_NODELAY alone is worth 4.26x, not 2.80x. The two changes are substitutes, not complements: TCP_NODELAY alone: 4.26x pipelining alone: 7.41x both together: 7.39x So: Pipelining alone captures the whole win. Adding the socket option on top of it is 0.997x, with overlapping ranges. Nothing. The reverse is not true: pipelining on top of NODELAY is still worth 1.73x. NODELAY does not substitute for pipelining. Pipelining substitutes for NODELAY, on throughput. What the socket option still does after pipelining is remove the stall, 16.1% of samples down to

Same only remote pools failed. Main XOA pool completed with no errors. { "data": { "mode": "delta", "reportWhen": "always", "backupReportTpl": "compactMjml", "hideSuccessfulItems": true }, "id": "1788224400022", "jobId": "c0e31730-7d45-4f3a-8733-8cea444b127d", "jobName": "Salem - Delta Backups", "message": "backup", "proxyId": "7840039e-a3cb-41ed-acee-3690f89a1426", "scheduleId": "ce3c801c-4384-4726-bda1-f6bae788e2f4", "start": 1788224400022, "status": "failure", "end": 1788224460395, "result": { "message": "fetch failed", "name": "TypeError", "stack": "TypeError: fetch failed\n at node:internal/deps/undici/undici:14976:13\n at AsyncResource.runInAsyncScope (node:async_hooks:214:14)\n at cb (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/util.js:355:42)\n at tryCatcher (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/util.js:16:23)\n at Promise._settlePromiseFromHandler (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:547:31)\n at Promise._settlePromise (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:604:18)\n at Promise._settlePromise0 (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:649:10)\n at Promise._settlePromises (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:725:18)\n at _drainQueueStep (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/async.js:93:12)\n at _drain

@ nikade yeah that's hot but would need explaining it as if it were for a 5 years old... asking for a friend :') did they overcome a bottleneck in VDI between SR copy ?

@ olivierlambert happy to see the CEO doing this kind of work, makes me proud to be one of your followers