Sync large data

[Zackri]

I have about 3 million files and folders sized (240G) on a local PC . Once the main job is synced only a thousand or so files change every day.

What I am trying to do is to sync the entire contents to a webhost via ftp and keep it updated as the files change on my local PC.

So what I need , is to transfer only the changes that happens on the files in the same time it changed on the destination .

I tried using GoodSync for this but it takes 15 hrs to analyses and by that time a number of files have changed.

Is there any software or string of batch files that would do this for me and utilize multiple connections for a ftp username?

- Greetings
Zack

 

[Ian]

Is there a way you can use something else other than FTP? It's not really suited to syncing files like that unfortunately. From what I understand, it'll always be fairly slow via FTP as it's hard to check so many files to see if they have changed.

Depending on the destination system you could use something like rsync (which works fine in Windows 11 using WSL).

 

[Zackri]

you are right, Onedrive would be a good choice, I am checking all ways now
rsync run on windows?

 

[Ian]

rsync run on windows?

Yep, if you're familiar with Linux you can install the Windows Subsystem for Linux (native to Windows 11) and use terminal commands like rsync

There's a tutorial on it here:

Install WSL

Install Windows Subsystem for Linux with the command, wsl --install. Use a Bash terminal on your Windows machine run by your preferred Linux distribution - Ubuntu, Debian, SUSE, Kali, Fedora, Pengwin, Alpine, and more are available.

docs.microsoft.com

 

[Zackri]

I appreciate the helpful info which you provided and I will take it into consideration