How to make import (>200M) faster? - 未名空间MITBBS历史存档

国际科技财经博客移民网络热点娱乐民生时事公众号

Redian新闻

>未名空间

>Database - 数据库

How to make import (>200M) faster?

How to make import (>200M) faster?# Database - 数据库

c*t2005-03-17 08:03

1 楼

import cvs file into one table (SQL Server), there are three indexes
created on the table. Is there any way to import faster?
Now it usually takes 30minutes to finish. about 10 minutes load data,
then build indexes (I guess).

s*n2005-03-17 08:03

2 楼

what's the current process?

【在 c**t 的大作中提到】

: import cvs file into one table (SQL Server), there are three indexes
: created on the table. Is there any way to import faster?
: Now it usually takes 30minutes to finish. about 10 minutes load data,
: then build indexes (I guess).

c*t2005-03-17 08:03

3 楼

App ftp csv file onto netshare;
scheduled job to pick the file and run dts
It takes about half an hour to finish the job. >60% to build indexes...
I have turned on "lock table", useless. :-(

【在 s******n 的大作中提到】

: what's the current process?

s*n2005-03-17 08:03

4 楼

Sounds pretty slow to me. Are you dropping the index and rebuilding after
the import?

【在 c**t 的大作中提到】

: App ftp csv file onto netshare;
: scheduled job to pick the file and run dts
: It takes about half an hour to finish the job. >60% to build indexes...
: I have turned on "lock table", useless. :-(

c*t2005-03-17 08:03

5 楼

It will take longer time to rebuild index...

【在 s******n 的大作中提到】

: Sounds pretty slow to me. Are you dropping the index and rebuilding after
: the import?

s*n2005-03-17 08:03

6 楼

DTS is supposed to be the fastest way to get data in. It's actually using the
SQL Server OLE DB IFastload interface to achieve best performance. Assuming
the process is IO-bounded, you may consider storing the data file, index
file, and log file on different disks.

【在 c**t 的大作中提到】

: It will take longer time to rebuild index...