2012年2月13日星期一

Batch insert (OLAP)

Hi can any one help me with the skeleton script (sample one)of running Bulk insert in batches...... I need to do it in batches as the input data is huge....

The logic is I have to insert thru bcp in fact table...

After that batch execution for 50,000 thousand record... wise... if any of the batch failes i need to identify and have to rerun from that point onwards..... this is OLAP thing...what's the data source?

50,000 ain't that much btw|||50,000 record comes in one batch that waht I meant...... so if there is 1 million record 20 batches will be there...

The source come from DB2 or so which we get it as source file... we create staging table for that in Sql Server... Now we need to do the rest porting data to Fact & Dimension tables|||what script?
you can use create a failover process in dts to allow x number of error rows to pass through the ETL and then you can clean them up the next day if you prefer. its part of the error reporting process in dts tasks.|||So the data is already in a table and you're trying to limit the impact to the logs...can you describe your process how are you building your warehouse..|||:eek: I didnt get u I never asked for error reporting stuff!!!!!|||http://www.winnetmag.com/article/articleid/42903/42903.html

I was trying to do the same way how the above article describe to solve such kinda stuff

没有评论:

发表评论