Using the Feeds Module for Bulk Import in Drupal 7

Here is how you can use the Feeds module to bulk import data using CSV files. This covers file uploads and taxonomies too. 

The requirement:

Bulk data upload of a single content type into Drupal 7. Data will include text, dates, files and tags/taxonomies. There may be multiple tags/taxonomy terms per vocabulary.

The solution:

Use Feeds module to import data from CSV files as Drupal nodes.

Required modules:

Modules/submodules to enable:

  • Feeds
  • Feeds Admin UI
  • Feeds Import
  • Feeds Tamper
  • Feeds Tamper Admin UI

Set up CSV:

Prepare the Excel file that you will convert to CSV - open up Excel and in the first row add the headers. Feeds importer requires a global ID to uniquely identify nodes. So the first column should contain this unique identifier. Name this column "guid". The other column headers can be anything you like. For example, if you want to import a list of files with title, abstract, some tags and taxonomies; you can name your columns filename, title, description, tags, categories etc. The column headers do not have to match content type fields.

Add data to the Excel file:

  • guid: make sure each row has a unique number.
  • filename: You will have to upload all the files you want to migrate, into your Drupal root directory. So put all your files in a directory called import-uploads and upload them to your Drupal root. Make sure that the filenames have the path from your Drupal root. So each filename should have "import-uploads/" prepended. E.g. "import-uploads/file1.pdf", "import-uploads/file2.avi", etc.
  • title/description: should have text, make sure that your title is less than 256 characters.
  • tags: add all tags (actual words for tags) on the same column for a specific row. Make sure that tags are semicolon separated. For example, if you want to tag the same node with "funny" and "entertaining" the entry should be "funny;entertaining".
  • taxonomies: This will probably require the most amount of effort. You have to find the taxonomy term ids for the terms you want to use for each row. Just like tags, if you have multiple taxonomy terms per node, make sure they are separated by semicolons. For example, if you have a taxonomy set up with terms "physics" with term id 22 and "math" with term id 25 and you want to classify the same node with both, the entry in the taxonomy column should be "22;25".

Save as CSV - save the Excel file in CSV format - make sure to quote all cells. Otherwise you will run into errors if any of the text (e.g. description) contains commas.

Set up Feeds Importer:

  • Navigate to Structure -> Feeds Importer -> Add Importer.
  • Give your importer a meaningful name like "bulk file upload" and click Submit.

On the next screen follow these directions:

  • Basic Settings: leave as is, no changes needed.
  • Fetched: click "change" and select the "File Upload" option and click save.
  • Fetched Fileupload: click settings, make sure to add all the file extentions you want to add, e.g. pdf, wav, doc, docx etc.
  • Parser: click "change", select the "CSV Parser" option and save.
  • CSV Parser: leave as is.
  • Processor: leave as is (Node Processor).
  • Node Processor: click settings, select "Replace existing node" for "Update exisiting node", select the Content Type you want to use for this import. You may choose to use an existing user as the author.
  • Click Save.
  • Node Processor Mapping: This is where you will map your CSV file headers to the Content Type fields. One by one, add the column headers of the CSV file to the mapping and choose the appropriate Content Type field to map to.
  • Click Save.
  • Now make some configuration changes for multi value inputs such as tags or taxonomies. To do this click on the "configure feeds tamper" link near the top of the Add Importer page.

For tags and taxonomies only, click "Add plugin". Select "Explode" as the "plugin to add", and in the "Configure Explodes" section change String Separator to a semicolon ";". This is the configuration that tells the feeds importer that tag/taxonomies have multiple values in the CSV file that are separated by semicolons. Once you are done, click the "Add" button. And then save the Feeds Tamper configurations.

Run the import:

  • Navigate to Structure -> Feeds Importer, and click on the "import" link.
  • Click on the Bulk File Upload link.
  • Upload your CSV file into your sites/default/files folder.
  • Upload your import-upload folder, containing all the files to be imported into your Drupal root.
  • Enter your CSV file's name in the File text field, with a the word "public://" prepended. So the entry should be something like "public://bulk_import.csv".
  • Click on the Import Button.

If you run into isses, you will see error messages at the top. Often times, these issues arise from typos in the CSV file, e.g. filename on CSV does not match filename on file system. If you come across issues like these, make the changes in the CSV and rerun the Import. You can keep on rerunning the import on the same CSV file as many times as you want.

Comments

Shakib, I tried this at home and it actually worked! At the risk of sounding like an idiot (very common with me and Drupal), I did have a question about "which directory is the root?", but I took a guess and put it under wamp/www/my_site/import-uploads and dang if it didn't run.

This is exactly what I needed and will save me a ton of time. Thank you for laying it out so even a crass newbie can follow it successfully.

Thanks one of the complet i have found.
Covering complex taxonomy and tag import

How to get terms ID?

Hi, yours is the only post I've found till now that has covered the taxonomy aspect of the node import. I am using XML as the file for my import (because I am exporting from another Drupal 6 site into this new Drupal 7 site). The problem I'm facing is that when I map the <Taxonomy> tag in the XML file to one of my taxonomy CCK field, it uses the term ID as it is and is not able to render the term id as the actual term. Eg., if the taxonomy tag holds the term id as 312, Drupal adds a term called 312 to my vocabulary and holds that value as the tag, instead of the term that actually has the term id 312.

I hope that makes sense. I am sure I am missing out some setting or a way to communicate to Drupal that this is a term id and not the value of the term. Would you be able to suggest me something for that?

Thanks a lot for this post though, it is really concise and just the thing.

I've already uploaded 1500 articles in English with unique IDs and now have more files with the same data in different languages (using the same ID). How can I import a second file so that it adds content under a language code (eg. FR, DE) but is linked to the original ID?

Hello,

After importing user i want to send welcome email to imported user.can you please give the tested suggestion for this.
if we can go for rules then pls export your rule and paste it here.
i am using feeds module for import user.

Thanks in advance

Thanks so much - this works great! Feeds is really powerful but like many things in the Drupal world it isn't easy to use and the documentation only goes so far.

So thanks for this really well thought out post. :D

I can't access the feeds node processor mapping page. It returns a blank page

Thank you. Your instructions were very clear. I wanted to point out that if you are importing a bunch of documents that only have a single taxonomy term that Feeds wills support using the text representation of that TID so that you don't have to actually look up the number.

how can i reanaming and reszie upload image? thanks very much!

Hi there, just want to say thanks!

Thanks to your advice I uploaded my first ever csv file on a site I'm making and that is something I've tried to get right for ages. So again, thanks! :D

amezing work

Thank you so much you helped me out of the problem I was having. This is the exact resource I was looking for. Thank you for sharing this. Works perfectly.
Underrated Books

Add new comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
Type the characters you see in this picture. (verify using audio)
Type the characters you see in the picture above; if you can't read them, submit the form and a new image will be generated. Not case sensitive.