Data Sets can be composed of one or many files and can be updated with subsequent file uploads. Once the information is configured properly, Avid will automatically add your data into the data set.
Files are added your Avid Platform through SFTP or a direct file upload. Accepted file types are .csv, .txt, and similar delimited text files. If you have an Excel file, please convert it to .csv before uploading.
After a file has been uploaded or delivered through SFTP, you can click the files link on the upload popup (below the upload box) or you can navigate to your Settings under Connections (the pill shaped selector is 1/3 of the way down the Connections page and has the options for Systems or Files -- choose Files). You will see a "+Add" in the data set column of the files list on the file or files you just uploaded.
Clicking on the "+ Add" button for a new file upload opens a new page for your Data Set Information. Here are the definitions for what you need to put in the fields:
- Data Set Name: This will populate with the name of the file your clicked on - it is editable so that you can use a more descriptive name and we suggest you pick a name that is descriptive of what is in the data set (i.e. Email Contact List_EOY Event)
- Data Category: choosing the correct category for the file brings up the relevant template for mapping and makes those columns available in Insights to use in your already built charts and reports so carefully consider where this data should surface for you to use - all data sets are available to use in Pathways if you have that feature in Avid (go here for information about Pathways).
- Activity Feed - Donor activities to use in Audience building
- Constituents - Contact and account information for constituents
- Cost - Custom built reporting for budgeting*
- Direct Mail Select - files used in segmenting direct mail sends
- Donation Form Submissions - results from online donation forms
- Email Contacts - subscriber information
- Email Form Submissions - results from online email forms
- Other - any file category not described here
- Transactions - Gift information for constituents
- Refresh Type: Append will add to the existing data set, while Replace will overwrite the existing data-for Append, you will have the chance to identify unique record columns
- Column(s) to Identify Unique Records: When appending files, you can select one or more columns to be used as a unique ID. If records with the same designated IDs have a record in a prior upload, the new data will be used as a replacement rather than create a new record. If no unique record identifiers are chosen from the provided columns, every records will always be appended. Your choice here will dictate how duplicate data records will be handled.
We strongly suggest that you at least choose an account ID option to eliminate multiple instances of the same account data - File Name Pattern: This identifies which set of files should be included in the data set
- Full File Name - If the file name will be the same every time, leave the field as is.
- Wildcard Matching - If the file name will vary from upload to upload, you have the option of entering specific words to look for such as "Direct Mail" or anything else. You can then add a % before and/or after the words which will allow it to match to those files. For example, if the file pattern were set as "direct_mail%", it would match a file named "direct_mail_september.csv" or "direct_mail_cultivation_20241010.csv".
- for more advanced rules, use a regular expression by starting with r' in this field and ending the expression with '.
- Ignore Unused Column Changes: keep this checked, especially if you are choosing to append your data. If your file structure will change (adding or deleting columns) reach out to support for additional instructions
When you press next, you will be given the option to exclude data from your file that you do not wish to include in your data set. Pressing next again allows you to map your file columns to the suggested Avid templates. File mapping will be covered in a separate article.
Creating a data set is easy and a great way to analyze information not included in your other connected systems. If you find you need additional help, search our knowledge base or feel free to reach out to support in the Avid Platform.