Monitoring transfer progress
Download source code for demo application: 06-progress.zip
In our previous examples we didn't monitored progress of transfers and just waited for them to finish. In interactive applications it's a good idea to give user some indication about how the progress goes, especially when transferring large amount of data, large files or batches. It's important to show user a progress of long-running operations. Otherwise they might think that the program hanged and kill it.
In non-interactive applications, such as automatic scheduled tasks, monitoring the progress isn't as important, but for long running tasks might still be benefical to log certain progress data for diagnostic purposes.
The Sftp class offers the TransferProgressChanged event, which is raised periodically during the operations. The SftpTransferProgressChangedEventArgs provided to the handler holds many useful status information:
-
Actionis the action being currently performed (Uploading,Downloading,Listing,Deleting). -
TransferStateis current state of the action being performed. -
BytesSinceLastEventis number of bytes transferred since last time the event was raised. You may use it to show only significant updates, especially when you are not overwriting old status information, such as when writing to a log file. -
BytesPerSecondis current transfer speed, if available. If not, 0 is returned. -
Data related to total transfer (entire batch for batch transfers):
BytesTotalis total number of bytes to be transferred.BytesTransferredis how many bytes was already transferred.FilesProcessedis number of files that has already been transferred or skipped.FilesTotalis total number of files in the batch.ProgressPercentageis decimal number from 0 to 100, number of percent completed total.
-
Data related to a file or directory (in case of listings) being currently processed:
CurrentFileBytesTotalis total number of bytes in current file.CurrentFileBytesTransferredis number of already transferred bytes in current file.CurrentFileProgressPercentageis decimal number from 0 to 100, number of percent completed in current file.SourceItemis the item being currently processed.TargetPathis target path of the item being currently processed.
Using the data available you can write fairly elaborate progress display. The following code will show both current and total progress in numerical form and in form of bar chart showing total progress. In progress, the output looks like this:
Logging in user tester...OK
Uploading files from C:\Windows\Web\Wallpaper\ThemeA...
Current: 94% of /ThemeA/img22.jpg (700.00 KB of 742.92 KB)
Total: 71% (2.27 MB of 3.17 MB), 508.16 KB/s
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░░░░░░░░░░░░░░░░░
The C# source code of full application looks like this:
using Rebex.Net;
using Rebex.IO;
const string ServerName = "localhost";
const string UserName = "tester";
const string Password = "password";
const string UploadDirectory = @"C:\Windows\Web\Wallpaper\ThemeA";
Rebex.Licensing.Key = "Get trial key at https://www.rebex.net/support/trial/";
using var ftp = new Sftp();
// Subscribe to the event to display progress information during file upload.
ftp.TransferProgressChanged += (s, e) => {
static string formatSize(long bytes) {
if (bytes >= 1024L * 1024 * 1024) return $"{bytes / (1024d * 1024 * 1024):N2} GB";
if (bytes >= 1024L * 1024) return $"{bytes / (1024d * 1024):N2} MB";
if (bytes >= 1024L) return $"{bytes / 1024d:N2} KB";
return $"{bytes:N0} B";
}
// Save the current cursor position
var pos = Console.GetCursorPosition();
// If there is current file progress information, display it. Otherwise, just move to the next line.
if (e.CurrentFileBytesTotal > 0) {
WriteFullLine($" Current: {e.CurrentFileProgressPercentage:N0}% of {e.TargetPath} ({formatSize(e.CurrentFileBytesTransferred)} of {formatSize(e.CurrentFileBytesTotal)})");
} else {
WriteFullLine( " Current: none");
}
// Display the overall upload progress
WriteFullLine($" Total: {e.ProgressPercentage:N0}% ({formatSize(e.BytesTransferred)} of {formatSize(e.BytesTotal)}), {formatSize(e.BytesPerSecond)}/s");
// Display a simple progress bar based on the overall progress percentage.
var fullCount = (int)(Console.WindowWidth * e.ProgressPercentage / 100);
Console.Write(new string('\x2593', fullCount)); // Dark shade block character
Console.Write(new string('\x2591', Console.WindowWidth - fullCount)); // Light shade block character
// Restore the cursor position to prepare for the next update.
Console.SetCursorPosition(pos.Left, pos.Top);
};
Console.Write("Connecting to SFTP server...");
await ftp.ConnectAsync(ServerName);
Console.WriteLine("OK");
Console.Write($"Logging in user {UserName}...");
await ftp.LoginAsync(UserName, Password);
Console.WriteLine("OK");
// Upload the directory recursively, overwriting all existing files on the server.
Console.WriteLine($"Uploading files from {UploadDirectory}...");
await ftp.UploadAsync(UploadDirectory, "/", TraversalMode.Recursive, TransferMethod.Copy, ActionOnExistingFiles.OverwriteAll);
// Move the cursor down to overwrite just the progress bar with the upload completed message, and clear the line to ensure it is fully overwritten.
Console.SetCursorPosition(0, Console.GetCursorPosition().Top + 2);
WriteFullLine("Upload completed.");
Console.Write("Disconnecting...");
await ftp.DisconnectAsync();
Console.WriteLine("OK");
static void WriteFullLine(string text = "") => Console.Write(text.PadRight(Console.WindowWidth).AsSpan(0, Console.WindowWidth));
The local function formatSize is there purely for user convenience, to choose appropriate unit for given amout of data.
The code relies heavily on the GetCursorPosition and SetCursorPosition methods to move the cursor around, because it overwrites the information on screen with new values. It looks pretty in interactive scenarios, but fails with System.IO.IOException if the output is being redirected (ie. using myapp.exe > logfile.txt).
To solve this problem, we have to implement the progress display logic twice, based on if the output is redirected or not. In case of redirection, we log simple progress information every 5 seconds. Take a look at the finished version of the progress logging code:
using Rebex.Net;
using Rebex.IO;
const string ServerName = "localhost";
const string UserName = "tester";
const string Password = "password";
const string UploadDirectory = @"C:\Windows\Web\Wallpaper\ThemeA";
const int LogInterval = 5000; // In milliseconds. Set to 0 to disable logging progress at intervals.
Rebex.Licensing.Key = "Get trial key at https://www.rebex.net/support/trial/";
DateTime? lastLogTime = null;
using var ftp = new Sftp();
// Set the maximum upload speed if specified (in kilobytes per second).
if (MaximumUploadSpeed > 0) ftp.MaxUploadSpeed = MaximumUploadSpeed;
// Subscribe to the event to display progress information during file upload.
ftp.TransferProgressChanged += (s, e) => {
if (!Console.IsOutputRedirected) {
displayFancyProgress(e);
} else if (LogInterval > 0 && e.BytesSinceLastEvent > 0) {
displaySimpleProgress(e);
}
static string formatSize(long bytes) {
if (bytes >= 1024L * 1024 * 1024) return $"{bytes / (1024d * 1024 * 1024):N2} GB";
if (bytes >= 1024L * 1024) return $"{bytes / (1024d * 1024):N2} MB";
if (bytes >= 1024L) return $"{bytes / 1024d:N2} KB";
return $"{bytes:N0} B";
}
static void displayFancyProgress(SftpTransferProgressChangedEventArgs e) {
// Save the current cursor position and use it to update the progress information
// in place without adding new lines to the console output.
var pos = Console.GetCursorPosition();
// If there is current file progress information, display it. Otherwise, just move to the next line.
if (e.CurrentFileBytesTotal > 0) {
WriteFullLine($" Current: {e.CurrentFileProgressPercentage:N0}% of {e.TargetPath} ({formatSize(e.CurrentFileBytesTransferred)} of {formatSize(e.CurrentFileBytesTotal)})");
} else {
WriteFullLine(" Current: none");
}
// Display the overall upload progress
WriteFullLine($" Total: {e.ProgressPercentage:N0}% ({formatSize(e.BytesTransferred)} of {formatSize(e.BytesTotal)}), {formatSize(e.BytesPerSecond)}/s");
// Display a simple progress bar based on the overall progress percentage.
var fullCount = (int)(Console.WindowWidth * e.ProgressPercentage / 100);
Console.Write(new string('\x2593', fullCount)); // Dark shade block character
Console.Write(new string('\x2591', Console.WindowWidth - fullCount)); // Light shade block character
// Restore the cursor position to prepare for the next update.
Console.SetCursorPosition(pos.Left, pos.Top);
}
void displaySimpleProgress(SftpTransferProgressChangedEventArgs e) {
// Display simple progress information without using cursor manipulation
// when output is redirected, and only log at specified intervals to avoid excessive logging.
if (e.ProgressPercentage == 100 || !lastLogTime.HasValue || (DateTime.Now - lastLogTime.Value).TotalMilliseconds >= LogInterval) {
if (e.CurrentFileBytesTotal > 0) Console.WriteLine($" Current: {e.CurrentFileProgressPercentage:N0}% of {e.TargetPath} ({formatSize(e.CurrentFileBytesTransferred)} of {formatSize(e.CurrentFileBytesTotal)})");
Console.WriteLine($" Total: {e.ProgressPercentage:N0}% ({formatSize(e.BytesTransferred)} of {formatSize(e.BytesTotal)}), {formatSize(e.BytesPerSecond)}/s");
lastLogTime = DateTime.Now;
}
}
};
Console.Write("Connecting to SFTP server...");
await ftp.ConnectAsync(ServerName);
Console.WriteLine("OK");
Console.Write($"Logging in user {UserName}...");
await ftp.LoginAsync(UserName, Password);
Console.WriteLine("OK");
// Upload the directory recursively, overwriting all existing files on the server.
Console.WriteLine($"Uploading files from {UploadDirectory}...");
await ftp.UploadAsync(UploadDirectory, "/", TraversalMode.Recursive, TransferMethod.Copy, ActionOnExistingFiles.OverwriteAll);
if (!Console.IsOutputRedirected) {
// Move the cursor down to overwrite just the progress bar with the upload completed message, and clear the line to ensure it is fully overwritten.
Console.SetCursorPosition(0, Console.GetCursorPosition().Top + 2);
}
WriteFullLine("Upload completed.");
Console.Write("Disconnecting...");
await ftp.DisconnectAsync();
Console.WriteLine("OK");
static void WriteFullLine(string text = "") => Console.Write(text.PadRight(Console.WindowWidth).AsSpan(0, Console.WindowWidth));
I split the progress logic into two local functions:
displayFancyProgressis the old code for interactive mode.displaySimpleProgressis the new code for redirection. The fairly complex structure of conditions for logging ensures, that:- progress logging can be disabled by setting
LogIntervalto zero, - nothing is added to log if no data were transferred from previous logging event, which is important especially for final stages of operation, when no files are transferred, but progress events are still raised,
- we log final 100% status,
- we log only every
LogIntervalmiliseconds and for the first time the event is raised.
- progress logging can be disabled by setting