Friday, 9 June 2017

How to Perform Incremental Load in Sqoop

Importing Incremental Data

You can also perform incremental imports using Sqoop. Incremental import is a technique that imports only the newly added rows in a table. It is required to add incremental , check-column , and last-value options to perform the incremental import.

incremental - Used by Sqoop to determine which rows are new. Legal values for this mode include append and lastmodified .

check-column - To provide the column that needs to checked the determine the candidate rows.

last-value - This is the maximum value of the last import run.


sqoop import 

--connect jdbc:mysql://localhost:3306/kalyan 
--username root 
--table sample 
-m 1
--incremental append 
--check-column id 
--last-value 1000

Share this article with your friends.


Related Posts Plugin for WordPress, Blogger...