Common SFTP Errors When Sending HRIS Data (And How to Fix Them)

Many companies rely on a secure file transfer protocol (SFTP) integration to move data from one platform to another, yet the process is notoriously finicky. Setting up the initial connection can be tricky and time-consuming, but the headaches don’t end there — SFTP errors are common and confusing.

The integration between your customer’s HRIS and your application relies on timely data, so it’s important to identify and resolve SFTP errors quickly before the systems become out of sync. This might mean working directly with your customers to troubleshoot — even if the root cause is within the data from your customer’s HRIS and not your application. But from your customer’s perspective, your application “doesn’t work,” and they will often look to you for a resolution. 

Whether it’s a data issue or connection error, you’re left trying to troubleshoot — often without an obvious way to pinpoint the problems. You might have access to error logs, but they’re difficult, if not impossible, to understand. Meanwhile, your customer is frustrated, and you have to meet compliance and SLA requirements.

If you’re not sure where to start, we’ve listed some common SFTP errors and how to identify them. We’ve also provided some information about an alternative that eliminates the need for SFTPs with a more stable, streamlined approach. 

Related: SFTP vs. API: Which integration method is best for employment data?

SFTP file errors

With SFTP, you’re relying on the extraction of data from your customer’s HRIS into a file. That file is securely moved to a specific directory, which is then “picked up” and imported by your application.

First and foremost, the file has to exist in the specified directory at the time the other application looks for it — whether that’s with each payroll run or on a different cadence. If your customer’s data doesn’t import as expected, that’s the first thing to check. 

Assuming that the file was in the correct location at the right time, you should next look to make sure that the file is in the right format. While it was probably configured correctly when you first set it up, changes in your customer’s HRIS might unexpectedly modify the file, causing the data import to fail. 

File type and format

Files are usually in a delimited format. Delimiters are special characters that indicate the separation of different pieces of data, like commas or tabs. You should check to make sure that the file has the right delimiter and that the delimiter is the same as it was in previous formats. Applications are typically coded with a delimiter in mind, so a file can’t switch between comma- or tab-delimited.

You should also check the file type to ensure it’s as expected. A comma-delimited file, for example, could be a .csv file or a .txt file. If the extension changes, your SFTP integration could fail.

Fields included 

SFTP imports rely on an exact match of columns in your customer’s file to fields in your application. While some applications may be coded to ignore extra fields, others may not. You’d either need to have your customer remove the extra fields in your file, map those fields in your application, or tell your application to ignore those fields. 

If you’d previously mapped a field and the field is removed from your file, that can also cause SFTP errors since the application is now receiving fewer fields than expected.

If your customer has created a custom field in their HRIS and wants to send it through your SFTP integration, make sure you or the customer create a matching custom field in your application before you start sending the data in your file. SFTP integrations will not create new fields in your application. 

File name and field names

Your SFTP integration might be looking for a specific file name (such as “dailyintegration.txt”). If the file name has changed, the import might not work. Sometimes, extracts from your customer’s HRIS might include a date and timestamp at the end of the file (such as “dailyintegration20240831.text). If so, you’d need to determine if your application can handle the changing date and timestamp, or if you or your customer will need to remove it before the import can occur. 

Within the file, some SFTP integrations are looking for exact field name matches between your file and the destination system. For example, if the name in the application is “First Name,” the name in your file must also be “First Name.” In some cases, the field is even case-sensitive, so if the field name is “first name,” the transfer will fail. 

File corruption

If something interrupts file creation (such as an unstable network), your customer might end up with a corrupted file. This can also happen if there are problems during the transfer process itself, such as memory issues.

You can usually detect file corruption if you open the file and it is blank or contains odd characters. 

UTF-8 encoding protocol

Unicode transformation format - 8 bits (UTF-8) is a common encoding protocol used by SFTP servers. If files contain special characters, UTF-8 ensures that data is accurately transmitted and received without corruption. If a file doesn’t follow a UTF-8 format and your application requires that format, the transfer will fail. The reverse can also be true: your customers are sending the file in UTF-8 format and your application can’t handle it, so it corrupts the file. 

SFTP data errors 

While file errors are tied to how the files are created, data errors are contained within the file. Sometimes, issues within individual records are enough to cause your SFTP integration to fail. 

Unless you’re getting clear error messages, the only way to find data errors is to open the file and search for the data that could be causing the issue. Given the size of these files, this can require a massive amount of effort.

Field mapping

You have to establish field mapping before you can complete a successful transfer. For example, in your customer’s HRIS, you might have a field “Start Date” and you need to map it to “Hire Date” in your application. 

If not mapped correctly, the SFTP integration might not work. If your customer incorrectly maps a field “First Name” to a field “Hire Date,” the data types are a mismatch (one is text, and one is a date field). Some SFTP integrations will reject this type of mapping and cause error messages or an SFTP failure. 

Duplicate data 

Your file integration will rely on a unique identifier for each record from your customer’s HRIS (such as the employee’s social security number or email). This is how the application can tell one employee record from another employee record. 

Unique identifiers cannot be duplicated in files. Some SFTP integrations may skip the duplicate records, but in other cases, it may cause the sync to fail. 

Data format

Mapping fields often requires a very specific format. Dates might need to be in either YYYY/MM/DD or MM/DD/YYYY. If your customer is sending any fields with numbers, a field may accept or reject decimals. 

If the SFTP integration fails, check the formatting of each field to make sure it matches the application’s requirements. 

Special characters

Some applications may be unable to handle special characters (like & or $). If your fields contain these characters, they will cause SFTP errors. Special characters may not cause complete SFTP errors, but may cause the specific fields not to be populated within your application, so your sync would be incomplete. 

Your customer would need to remove these characters so your data can sync properly. 

Terminated or inactive employee data 

Many applications that sync from an HRIS want only active employee data. You’d need to ensure that your customer filters the HRIS files to remove terminated or inactive employee records before they’re sent via SFTP. 

If your customer sends a file containing terminated or inactive employees, it can cause a lot of issues. Your application may not be able to handle the records or it can create “junk” records in your application that you’d need to clean up. 

SFTP connection errors 

SFTP failure can also happen when there’s a connection issue between the source (your customer’s HRIS) and the destination (your application). You may need to involve someone from your customer’s technical support team to troubleshoot these issues. 

Destination folder

When your customer sends a file from their HRIS, it has to be placed in a destination folder for the application to ingest. If the destination folder doesn’t have the correct permissions (such as being read-only versus read-write), the file won’t transfer.

Your customer may also have issues within the folder itself. In some cases, the destination folder has to be left blank to ensure that data is ingested. Additionally, if your customer sends too many files at once — especially if they have the same file name — the sync may fail. 

Permissions

The file transfer itself may be set up as a scheduled job, which your customer needs to run with administrator access. If it’s run with a specific employee’s account, and that employee leaves without granting access, it can be difficult to transfer permission for the scheduled job to a new employee. 

Additionally, your customer’s firewall may prevent sensitive data from leaving any internal systems. Your IT department may need to disable the firewall to allow the SFTP sync to occur. 

SSH key pairing

SSH key pairing is an authentication method for SFTP that’s more secure than using a password. On your customer’s SFTP server, your customer will have a public key that acts like a “lock.” During the connection, a private key is used to authenticate and “unlock” the server. The combination of keys — the public key and private key — is the key pair. 

SSH keys can be tricky to manage, especially if your customer is doing this manually. Keys are stored in .ppk files, which may require downloading additional software to generate the files. Keys can be “orphaned” if they’re issued to employees who leave the organization and IT staff forgets to terminate the keys. Keys can also be blocked by firewalls, permissions, or server configuration. 

Erroneous actions

SFTP has many operations, such as “put” (upload), “get” (download), “rename” and more. Your customer’s SFTP needs to use the “put” action only. If you include something else (like “rename”), it may cause the sync to fail. 

Unsupported SFTP protocol

Unsupported SFTP protocol errors happen when there’s a mismatch or incompatibility between the source and the destination. Your customer’s server may only accept certain configurations. 

SFTP has multiple versions, and they’re not all compatible with each other. Additionally, SFTP runs over SSH which also has multiple versions (SSH1 versus SSH2), which can cause incompatibilities. Your customer’s security settings may also cause protocol issues. 

Related: Improving Business KPIs with SFTP Scalability and Automation

HRIS integrations: An alternative to SFTP

Even after you’ve set up your SFTP (which can be difficult by itself) and it’s running as expected, you can experience issues down the road. If there are changes in the underlying technology, such as your customer’s HRIS or in your own application, it can cause SFTP errors. You have control over your own app, but changes to your customers’ various HRIS systems are unpredictable, meaning maintaining a healthy SFTP connection is an ongoing, intensive effort.

To make matters more difficult, each HRIS has its own quirks and data schemas. Each customer will require the configuration of their own SFTP connection, but these connections can’t follow a routine protocol because each HRIS will have different data formats and connection requirements. 

By contrast, HRIS integrations powered by APIs offer an automated, standardized alternative to complex SFTP connections and a way to avoid common SFTP errors altogether. With Finch’s Unified Employment API, you can build a single API connection to access your customers’ data across hundreds of HRIS and payroll systems. 

Finch: The HRIS and Payroll Integration Solution

With Finch’s API integrations, your application can seamlessly connect to the various HRIS and payroll systems your customers use, without having to build individual SFTP connections for each customer. Better yet, the data is automatically standardized, meaning it is delivered to your application in the appropriate format, regardless of how it’s configured in the HRIS or payroll system. The data is also sent daily or weekly, as opposed to just once per pay cycle.

Finch’s integrations can be accessed in one of two ways — either through an API integration with Finch, or through Finch Flatfile, now in beta. Flatfile provides all the benefits of our API, but still delivers files via SFTP.

Finch has access to the largest network of HRIS and payroll systems, providing deep integrations without the headaches of SFTP. Companies using Finch can take advantage of Finch’s Unified Employment API and allow their customers to integrate their HRIS and payroll platforms seamlessly. 

Learn more about Finch by signing up for free or scheduling a call.