显示标签为“events”的博文。显示所有博文
显示标签为“events”的博文。显示所有博文

2012年3月6日星期二

BCP error - missing data

After loading the BCP files that are created during the trigger/
reporting events I've noticed that the data in the table is missing
records. I've also noticed that the missing records (records in the
table but not in the BCP out files) seem to occur in contiguous
blocks. Since the complete set of records exists in the table, I
assume this points to an issue in the way the TableUpdate script/
Triggers interact with the system. But i tried to take out the bcp
procedure, do test on trigger, then no data missing, So, I think the
problem is still on bcp part. Could you help me with that?

CREATE TABLE [EventUpdate] (
[id] [int] NOT NULL ,
[eventid] [int] NOT NULL ,
[sequenceid] [int] NOT NULL ,
[UpdatePass] [int] NULL
) ON [PRIMARY]
GO
create trigger trgEventUpdate on EventLog For Insert,Update as
insert into EventUpdate (id,eventid,sequenceid) select ins.id,
ins.eventid,ins.sequenceid from inserted ins

bcp script:

bcp "select a.* from w..eventlog a, w..eventupdate b where
a.eventid=b.eventid and a.sequenceid=b.sequenceid and b.eventid<>-1
and b.sequenceid<>-1 and b.updatepass=1" queryout 30sec-%TFN_NOW%.wrk -
U <userwithaccess-P <password-S doserver -f EventLog.fmt

Thanks in advance for your reply!(danceli@.gmail.com) writes:

Quote:

Originally Posted by

After loading the BCP files that are created during the trigger/
reporting events I've noticed that the data in the table is missing
records. I've also noticed that the missing records (records in the
table but not in the BCP out files) seem to occur in contiguous
blocks. Since the complete set of records exists in the table, I
assume this points to an issue in the way the TableUpdate script/
Triggers interact with the system. But i tried to take out the bcp
procedure, do test on trigger, then no data missing, So, I think the
problem is still on bcp part. Could you help me with that?


Please, could you take that again, and more slowly this time? Keep in
mind that people reading this newsgroup does not know about your system.

You are missing data but where? You appear to extract data with BCP, do
you load it anywhere else?

One thing I can note:

Quote:

Originally Posted by

create trigger trgEventUpdate on EventLog For Insert,Update as
insert into EventUpdate (id,eventid,sequenceid) select ins.id,
ins.eventid,ins.sequenceid from inserted ins
>
>bcp "select a.* from w..eventlog a, w..eventupdate b where
>a.eventid=b.eventid and a.sequenceid=b.sequenceid and b.eventid<>-1
>and b.sequenceid<>-1 and b.updatepass=1" queryout 30sec-%TFN_NOW%.wrk -
>U <userwithaccess-P <password-S doserver -f EventLog.fmt


The BCP query has a condition on the column UpdatePass which is not
set in the trigger. So from what you have posted, I would not expect
anything at all to go in the BCP file.

--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se
Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pr...oads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodin...ions/books.mspx|||1.

Quote:

Originally Posted by

You are missing data but where? You appear to extract data withBCP, do
you load it anywhere else?


On table Eventlog, there are missing data.

2.

Quote:

Originally Posted by

Quote:

Originally Posted by

create trigger trgEventUpdate on EventLog For Insert,Update as
insert into EventUpdate (id,eventid,sequenceid) select ins.id,
ins.eventid,ins.sequenceid from inserted ins


>

Quote:

Originally Posted by

bcp"select a.* from w..eventlog a, w..eventupdate b where
a.eventid=b.eventid and a.sequenceid=b.sequenceid and b.eventid<>-1
and b.sequenceid<>-1 and b.updatepass=1" queryout 30sec-%TFN_NOW%.wrk -
U <userwithaccess-P <password-S doserver -f EventLog.fmt


>
TheBCPquery has a condition on the column UpdatePass which is not
set in the trigger. So from what you have posted, I would not expect
anything at all to go in theBCPfile.


sorry, i didn't clearly explain it. i did set the UpdataPass:

CREATE TABLE [EventLogUpdate] (
[id] [int] NOT NULL ,
[eventid] [int] NOT NULL ,
[sequenceid] [int] NOT NULL ,
[UpdatePass] [int] NULL
) ON [PRIMARY]
GO
create trigger trgEventUpdate on EventLog For Insert,Update as
insert into EventLogUpdate (id,eventid,sequenceid) select ins.id,
ins.eventid,ins.sequenceid from inserted ins
GO|||(danceli@.gmail.com) writes:

Quote:

Originally Posted by

On table Eventlog, there are missing data.


But it was the EventLog table you had the trigger on? If data is
missing in EventLog what has the trigger or the BCP stuff to do with it?

Quote:

Originally Posted by

sorry, i didn't clearly explain it. i did set the UpdataPass:
>
CREATE TABLE [EventLogUpdate] (
[id] [int] NOT NULL ,
[eventid] [int] NOT NULL ,
[sequenceid] [int] NOT NULL ,
[UpdatePass] [int] NULL
) ON [PRIMARY]
GO
create trigger trgEventUpdate on EventLog For Insert,Update as
insert into EventLogUpdate (id,eventid,sequenceid) select ins.id,
ins.eventid,ins.sequenceid from inserted ins
GO


I still can't see any update of UpdatePass?

It's your call. If you don't want to explain what you are doing, that
is alright. But unfortunately it is difficult to answer your questions
in this case.

--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se
Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pr...oads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodin...ions/books.mspx

BCP error - missing data

After loading the BCP files that are created during the trigger/
reporting events I've noticed that the data in the table is missing
records. I've also noticed that the missing records (records in the
table but not in the BCP out files) seem to occur in contiguous
blocks. Since the complete set of records exists in the table, I
assume this points to an issue in the way the TableUpdate script/
Triggers interact with the system. But i tried to take out the bcp
procedure, do test on trigger, then no data missing, So, I think the
problem is still on bcp part. Could you help me with that?

CREATE TABLE [EventUpdate] (
[id] [int] NOT NULL ,
[eventid] [int] NOT NULL ,
[sequenceid] [int] NOT NULL ,
[UpdatePass] [int] NULL
) ON [PRIMARY]
GO
create trigger trgEventUpdate on EventLog For Insert,Update as
insert into EventUpdate (id,eventid,sequenceid) select ins.id,
ins.eventid,ins.sequenceid from inserted ins

bcp script:

bcp "select a.* from w..eventlog a, w..eventupdate b where
a.eventid=b.eventid and a.sequenceid=b.sequenceid and b.eventid<>-1
and b.sequenceid<>-1 and b.updatepass=1" queryout 30sec-%TFN_NOW%.wrk -
U <userwithaccess-P <password-S doserver -f EventLog.fmt

Thanks in advance for your reply!Danceli,

Are your triggers set-based?

-- Bill

<danceli@.gmail.comwrote in message
news:1170212352.130762.295120@.m58g2000cwm.googlegr oups.com...

Quote:

Originally Posted by

After loading the BCP files that are created during the trigger/
reporting events I've noticed that the data in the table is missing
records. I've also noticed that the missing records (records in the
table but not in the BCP out files) seem to occur in contiguous
blocks. Since the complete set of records exists in the table, I
assume this points to an issue in the way the TableUpdate script/
Triggers interact with the system. But i tried to take out the bcp
procedure, do test on trigger, then no data missing, So, I think the
problem is still on bcp part. Could you help me with that?
>
CREATE TABLE [EventUpdate] (
[id] [int] NOT NULL ,
[eventid] [int] NOT NULL ,
[sequenceid] [int] NOT NULL ,
[UpdatePass] [int] NULL
) ON [PRIMARY]
GO
create trigger trgEventUpdate on EventLog For Insert,Update as
insert into EventUpdate (id,eventid,sequenceid) select ins.id,
ins.eventid,ins.sequenceid from inserted ins
>
>
>
bcp script:
>
bcp "select a.* from w..eventlog a, w..eventupdate b where
a.eventid=b.eventid and a.sequenceid=b.sequenceid and b.eventid<>-1
and b.sequenceid<>-1 and b.updatepass=1" queryout 30sec-%TFN_NOW%.wrk -
U <userwithaccess-P <password-S doserver -f EventLog.fmt
>
Thanks in advance for your reply!
>

2012年2月18日星期六

BCP - BULK INSERT - Memory mapped files

Hi,
I have an application server which receives several thousand events per
second. And it needs to store that data in an SQL server running on a
separate server connected thru network to the application server. I would
like to know the fastest and the efficient way to store the data in to SQL
server tables.
I looked in to BCP and BULK INSERT, but the question is how do I move the
data file(presumably created by the application server) to the SQL server to
use with BULK INSERT? I can probably create the file locally on the
application server, and use bcp_execute etc programatically, but is this the
best way? Since the data is already in the application memory, is it possible
make bcp_execute access the in-memory data, using memory-mapped file or some
such thing?
Thank you for your help, Srini
Check SqlBulkCopy class in msdn or .Net documentation.
You can feed it with any object (WriteToServer method) implementing
IDataReader
HTH
"Srini" <Srini@.discussions.microsoft.com> wrote in message
news:AB6E2972-4C04-47AE-BA3D-1BF7EEE827A5@.microsoft.com...
> Hi,
> I have an application server which receives several thousand events per
> second. And it needs to store that data in an SQL server running on a
> separate server connected thru network to the application server. I would
> like to know the fastest and the efficient way to store the data in to SQL
> server tables.
> I looked in to BCP and BULK INSERT, but the question is how do I move the
> data file(presumably created by the application server) to the SQL server
> to
> use with BULK INSERT? I can probably create the file locally on the
> application server, and use bcp_execute etc programatically, but is this
> the
> best way? Since the data is already in the application memory, is it
> possible
> make bcp_execute access the in-memory data, using memory-mapped file or
> some
> such thing?
> Thank you for your help, Srini
|||I am not using .Net Framework in my code. I need a solution using ODBC or BCP
API. etc.
Thanks for your help.
"AlexS" wrote:

> Check SqlBulkCopy class in msdn or .Net documentation.
> You can feed it with any object (WriteToServer method) implementing
> IDataReader
> HTH
> "Srini" <Srini@.discussions.microsoft.com> wrote in message
> news:AB6E2972-4C04-47AE-BA3D-1BF7EEE827A5@.microsoft.com...
>
>
|||Look up bcp_bind and bcp_sendrow in MSDN
"Srini" <Srini@.discussions.microsoft.com> wrote in message
news:62EB5419-1361-40BA-A66E-40EC2817A5BC@.microsoft.com...[vbcol=seagreen]
>I am not using .Net Framework in my code. I need a solution using ODBC or
>BCP
> API. etc.
> Thanks for your help.
> "AlexS" wrote:

BCP - BULK INSERT - Memory mapped files

Hi,
I have an application server which receives several thousand events per
second. And it needs to store that data in an SQL server running on a
separate server connected thru network to the application server. I would
like to know the fastest and the efficient way to store the data in to SQL
server tables.
I looked in to BCP and BULK INSERT, but the question is how do I move the
data file(presumably created by the application server) to the SQL server to
use with BULK INSERT? I can probably create the file locally on the
application server, and use bcp_execute etc programatically, but is this the
best way? Since the data is already in the application memory, is it possibl
e
make bcp_execute access the in-memory data, using memory-mapped file or some
such thing?
Thank you for your help, SriniCheck SqlBulkCopy class in msdn or .Net documentation.
You can feed it with any object (WriteToServer method) implementing
IDataReader
HTH
"Srini" <Srini@.discussions.microsoft.com> wrote in message
news:AB6E2972-4C04-47AE-BA3D-1BF7EEE827A5@.microsoft.com...
> Hi,
> I have an application server which receives several thousand events per
> second. And it needs to store that data in an SQL server running on a
> separate server connected thru network to the application server. I would
> like to know the fastest and the efficient way to store the data in to SQL
> server tables.
> I looked in to BCP and BULK INSERT, but the question is how do I move the
> data file(presumably created by the application server) to the SQL server
> to
> use with BULK INSERT? I can probably create the file locally on the
> application server, and use bcp_execute etc programatically, but is this
> the
> best way? Since the data is already in the application memory, is it
> possible
> make bcp_execute access the in-memory data, using memory-mapped file or
> some
> such thing?
> Thank you for your help, Srini|||I am not using .Net Framework in my code. I need a solution using ODBC or BC
P
API. etc.
Thanks for your help.
"AlexS" wrote:

> Check SqlBulkCopy class in msdn or .Net documentation.
> You can feed it with any object (WriteToServer method) implementing
> IDataReader
> HTH
> "Srini" <Srini@.discussions.microsoft.com> wrote in message
> news:AB6E2972-4C04-47AE-BA3D-1BF7EEE827A5@.microsoft.com...
>
>|||Look up bcp_bind and bcp_sendrow in MSDN
"Srini" <Srini@.discussions.microsoft.com> wrote in message
news:62EB5419-1361-40BA-A66E-40EC2817A5BC@.microsoft.com...[vbcol=seagreen]
>I am not using .Net Framework in my code. I need a solution using ODBC or
>BCP
> API. etc.
> Thanks for your help.
> "AlexS" wrote:
>