Skip to main content

Monitoring transfer progress

Download sample

Download source code for demo application: 06-progress.zip

In our previous examples we didn't monitored progress of transfers and just waited for them to finish. In interactive applications it's a good idea to give user some indication about how the progress goes, especially when transferring large amount of data, large files or batches. It's important to show user a progress of long-running operations. Otherwise they might think that the program hanged and kill it.

In non-interactive applications, such as automatic scheduled tasks, monitoring the progress isn't as important, but for long running tasks might still be benefical to log certain progress data for diagnostic purposes.

The Sftp class offers the TransferProgressChanged event, which is raised periodically during the operations. The SftpTransferProgressChangedEventArgs provided to the handler holds many useful status information:

  • Action is the action being currently performed (Uploading, Downloading, Listing, Deleting).

  • TransferState is current state of the action being performed.

  • BytesSinceLastEvent is number of bytes transferred since last time the event was raised. You may use it to show only significant updates, especially when you are not overwriting old status information, such as when writing to a log file.

  • BytesPerSecond is current transfer speed, if available. If not, 0 is returned.

  • Data related to total transfer (entire batch for batch transfers):

    • BytesTotal is total number of bytes to be transferred.
    • BytesTransferred is how many bytes was already transferred.
    • FilesProcessed is number of files that has already been transferred or skipped.
    • FilesTotal is total number of files in the batch.
    • ProgressPercentage is decimal number from 0 to 100, number of percent completed total.
  • Data related to a file or directory (in case of listings) being currently processed:

    • CurrentFileBytesTotal is total number of bytes in current file.
    • CurrentFileBytesTransferred is number of already transferred bytes in current file.
    • CurrentFileProgressPercentage is decimal number from 0 to 100, number of percent completed in current file.
    • SourceItem is the item being currently processed.
    • TargetPath is target path of the item being currently processed.

    Using the data available you can write fairly elaborate progress display. The following code will show both current and total progress in numerical form and in form of bar chart showing total progress. In progress, the output looks like this:

Logging in user tester...OK
Uploading files from C:\Windows\Web\Wallpaper\ThemeA...
Current: 94% of /ThemeA/img22.jpg (700.00 KB of 742.92 KB)
Total: 71% (2.27 MB of 3.17 MB), 508.16 KB/s

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░░░░░░░░░░░░░░░░░

The C# source code of full application looks like this:

using Rebex.Net;
using Rebex.IO;

const string ServerName = "localhost";
const string UserName = "tester";
const string Password = "password";
const string UploadDirectory = @"C:\Windows\Web\Wallpaper\ThemeA";

Rebex.Licensing.Key = "Get trial key at https://www.rebex.net/support/trial/";

using var ftp = new Sftp();

// Subscribe to the event to display progress information during file upload.
ftp.TransferProgressChanged += (s, e) => {

static string formatSize(long bytes) {
if (bytes >= 1024L * 1024 * 1024) return $"{bytes / (1024d * 1024 * 1024):N2} GB";
if (bytes >= 1024L * 1024) return $"{bytes / (1024d * 1024):N2} MB";
if (bytes >= 1024L) return $"{bytes / 1024d:N2} KB";
return $"{bytes:N0} B";
}

// Save the current cursor position
var pos = Console.GetCursorPosition();

// If there is current file progress information, display it. Otherwise, just move to the next line.
if (e.CurrentFileBytesTotal > 0) {
WriteFullLine($" Current: {e.CurrentFileProgressPercentage:N0}% of {e.TargetPath} ({formatSize(e.CurrentFileBytesTransferred)} of {formatSize(e.CurrentFileBytesTotal)})");
} else {
WriteFullLine( " Current: none");
}

// Display the overall upload progress
WriteFullLine($" Total: {e.ProgressPercentage:N0}% ({formatSize(e.BytesTransferred)} of {formatSize(e.BytesTotal)}), {formatSize(e.BytesPerSecond)}/s");

// Display a simple progress bar based on the overall progress percentage.
var fullCount = (int)(Console.WindowWidth * e.ProgressPercentage / 100);
Console.Write(new string('\x2593', fullCount)); // Dark shade block character
Console.Write(new string('\x2591', Console.WindowWidth - fullCount)); // Light shade block character

// Restore the cursor position to prepare for the next update.
Console.SetCursorPosition(pos.Left, pos.Top);
};

Console.Write("Connecting to SFTP server...");
await ftp.ConnectAsync(ServerName);
Console.WriteLine("OK");

Console.Write($"Logging in user {UserName}...");
await ftp.LoginAsync(UserName, Password);
Console.WriteLine("OK");

// Upload the directory recursively, overwriting all existing files on the server.
Console.WriteLine($"Uploading files from {UploadDirectory}...");
await ftp.UploadAsync(UploadDirectory, "/", TraversalMode.Recursive, TransferMethod.Copy, ActionOnExistingFiles.OverwriteAll);

// Move the cursor down to overwrite just the progress bar with the upload completed message, and clear the line to ensure it is fully overwritten.
Console.SetCursorPosition(0, Console.GetCursorPosition().Top + 2);
WriteFullLine("Upload completed.");

Console.Write("Disconnecting...");
await ftp.DisconnectAsync();
Console.WriteLine("OK");

static void WriteFullLine(string text = "") => Console.Write(text.PadRight(Console.WindowWidth).AsSpan(0, Console.WindowWidth));
info

The local function formatSize is there purely for user convenience, to choose appropriate unit for given amout of data.

The code relies heavily on the GetCursorPosition and SetCursorPosition methods to move the cursor around, because it overwrites the information on screen with new values. It looks pretty in interactive scenarios, but fails with System.IO.IOException if the output is being redirected (ie. using myapp.exe > logfile.txt).

To solve this problem, we have to implement the progress display logic twice, based on if the output is redirected or not. In case of redirection, we log simple progress information every 5 seconds. Take a look at the finished version of the progress logging code:

using Rebex.Net;
using Rebex.IO;

const string ServerName = "localhost";
const string UserName = "tester";
const string Password = "password";
const string UploadDirectory = @"C:\Windows\Web\Wallpaper\ThemeA";
const int LogInterval = 5000; // In milliseconds. Set to 0 to disable logging progress at intervals.

Rebex.Licensing.Key = "Get trial key at https://www.rebex.net/support/trial/";

DateTime? lastLogTime = null;

using var ftp = new Sftp();

// Set the maximum upload speed if specified (in kilobytes per second).
if (MaximumUploadSpeed > 0) ftp.MaxUploadSpeed = MaximumUploadSpeed;

// Subscribe to the event to display progress information during file upload.
ftp.TransferProgressChanged += (s, e) => {
if (!Console.IsOutputRedirected) {
displayFancyProgress(e);
} else if (LogInterval > 0 && e.BytesSinceLastEvent > 0) {
displaySimpleProgress(e);
}

static string formatSize(long bytes) {
if (bytes >= 1024L * 1024 * 1024) return $"{bytes / (1024d * 1024 * 1024):N2} GB";
if (bytes >= 1024L * 1024) return $"{bytes / (1024d * 1024):N2} MB";
if (bytes >= 1024L) return $"{bytes / 1024d:N2} KB";
return $"{bytes:N0} B";
}

static void displayFancyProgress(SftpTransferProgressChangedEventArgs e) {
// Save the current cursor position and use it to update the progress information
// in place without adding new lines to the console output.
var pos = Console.GetCursorPosition();

// If there is current file progress information, display it. Otherwise, just move to the next line.
if (e.CurrentFileBytesTotal > 0) {
WriteFullLine($" Current: {e.CurrentFileProgressPercentage:N0}% of {e.TargetPath} ({formatSize(e.CurrentFileBytesTransferred)} of {formatSize(e.CurrentFileBytesTotal)})");
} else {
WriteFullLine(" Current: none");
}

// Display the overall upload progress
WriteFullLine($" Total: {e.ProgressPercentage:N0}% ({formatSize(e.BytesTransferred)} of {formatSize(e.BytesTotal)}), {formatSize(e.BytesPerSecond)}/s");

// Display a simple progress bar based on the overall progress percentage.
var fullCount = (int)(Console.WindowWidth * e.ProgressPercentage / 100);
Console.Write(new string('\x2593', fullCount)); // Dark shade block character
Console.Write(new string('\x2591', Console.WindowWidth - fullCount)); // Light shade block character

// Restore the cursor position to prepare for the next update.
Console.SetCursorPosition(pos.Left, pos.Top);
}

void displaySimpleProgress(SftpTransferProgressChangedEventArgs e) {
// Display simple progress information without using cursor manipulation
// when output is redirected, and only log at specified intervals to avoid excessive logging.
if (e.ProgressPercentage == 100 || !lastLogTime.HasValue || (DateTime.Now - lastLogTime.Value).TotalMilliseconds >= LogInterval) {
if (e.CurrentFileBytesTotal > 0) Console.WriteLine($" Current: {e.CurrentFileProgressPercentage:N0}% of {e.TargetPath} ({formatSize(e.CurrentFileBytesTransferred)} of {formatSize(e.CurrentFileBytesTotal)})");
Console.WriteLine($" Total: {e.ProgressPercentage:N0}% ({formatSize(e.BytesTransferred)} of {formatSize(e.BytesTotal)}), {formatSize(e.BytesPerSecond)}/s");
lastLogTime = DateTime.Now;
}
}
};

Console.Write("Connecting to SFTP server...");
await ftp.ConnectAsync(ServerName);
Console.WriteLine("OK");

Console.Write($"Logging in user {UserName}...");
await ftp.LoginAsync(UserName, Password);
Console.WriteLine("OK");

// Upload the directory recursively, overwriting all existing files on the server.
Console.WriteLine($"Uploading files from {UploadDirectory}...");
await ftp.UploadAsync(UploadDirectory, "/", TraversalMode.Recursive, TransferMethod.Copy, ActionOnExistingFiles.OverwriteAll);

if (!Console.IsOutputRedirected) {
// Move the cursor down to overwrite just the progress bar with the upload completed message, and clear the line to ensure it is fully overwritten.
Console.SetCursorPosition(0, Console.GetCursorPosition().Top + 2);
}
WriteFullLine("Upload completed.");

Console.Write("Disconnecting...");
await ftp.DisconnectAsync();
Console.WriteLine("OK");

static void WriteFullLine(string text = "") => Console.Write(text.PadRight(Console.WindowWidth).AsSpan(0, Console.WindowWidth));

I split the progress logic into two local functions:

  • displayFancyProgress is the old code for interactive mode.
  • displaySimpleProgress is the new code for redirection. The fairly complex structure of conditions for logging ensures, that:
    • progress logging can be disabled by setting LogInterval to zero,
    • nothing is added to log if no data were transferred from previous logging event, which is important especially for final stages of operation, when no files are transferred, but progress events are still raised,
    • we log final 100% status,
    • we log only every LogInterval miliseconds and for the first time the event is raised.