Unix Technical Forum

Re: Antw: COPY FROM

This is a discussion on Re: Antw: COPY FROM within the pgsql Interfaces odbc forums, part of the PostgreSQL category; --> Hi Thomas, That's the way I have it working already. With 2,700,000 records it takes quite a lot of ...


Go Back   Unix Technical Forum > Database Server Software > PostgreSQL > pgsql Interfaces odbc

Register FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 04-16-2008, 01:50 AM
Bart Degryse
 
Posts: n/a
Default Re: Antw: COPY FROM

Hi Thomas,
That's the way I have it working already. With 2,700,000 records it takes quite a lot of time though. According to the PostgreSQL manual bulk uploads perform better with the COPY FROM statement. If only I could get it to work...
Thanks anyway.

>>> "Thomas Holschen" <Thomas.Holschen@hela-food.de> 2006-02-08 17:30 >>>


Hi Bart,

Just create an ODBC Entry on your local computer,
add a Refernce to ActiveX Dataobjects 2.8 oder 2.7 to your vbp
Projekt,
open an ADODB.connection to the Server and send SQL-Insert Statements.

Little Example
Private Function Insert()
Dim DBS As New ADODB.Connection
Dim SQLString As String
DBS.Open "Provider=MSDASQL.1;Persist Security Info=False;Extended
Properties=DSN=YourODBCDatabaseName;"

SQLString = "Insert into SomeTable (Field1,Field2,Field3) Values
('111','aaa','bbb')"
DBS.Execute SQLString
End Function

Hope that helps... Your Mail is a little bit "unreadable" ;-)

regards, Thomas.





--

_____________________________________________
Hela Gewürzwerk Hermann Laue GmbH & Co.KG
EDV
Thomas Holschen
Beimoorweg 11
22926 Ahrensburg

Tel. : +49 4102/496-381
http://www.hela-food.de
_____________________________________________

>>> "Bart Degryse" <Bart.Degryse@indicator.be> schrieb am Mittwoch, 8.

Februar 2006
um 11:03 in Nachricht <s3e9d02f.034@webaccess.indicator.be>:

> Dear,
> I need to do a bulk upload (2,600,000 records) of data into a

PostgreSQL
> (v8.0.3) table. I'm trying to achieve this from Visual Basic with ADO

and
> psqlODBC (v8.1.2) but I can't get it working. Currently my code looks

like
> this. Dim conn As New ADODB.Connection Dim query As String

'DSN less
> connection query = "DRIVER={PostgreSQL
>

Unicode};SERVER=10.100.1.24;PORT=2345;DATABASE=big db;BoolsAsChar=0;TrueIsMin
> us1=1;Debug=0;CommLog=0" conn.CursorLocation = adUseClient

conn.Open
> query, "bad", "xxxxxxxx" query = "COPY dunn_main (duns, company,
> company_short, zip, phone, employee_number, legal_id, sic_id,

source_id) " & _
> "FROM STDIN WITH NULL AS 'NULL' DELIMITER AS ','"

conn.Execute
> query, , adCmdText + adExecuteNoRecords + adAsyncExecute
> In the driver logging I can see that it's waiting for the data now,

but I
> can't really figure out how to deliver it. Since the source data (as

a text
> file with fixed length fields) is only available on client side and

needs
> some processing before being ready to import I'm using something like

this to
> prepare the data:
> Private Type Dunn_Record CO_NAME As String * 90

PCODE As
> String * 8 DUNS As String * 9 EMPS_COMP As String * 9

LE
> As String * 2 L As String * 1 TEL_NBR As String * 14


> US72 As String * 4 crlf As String * 2 End Type Dim

record As
> Dunn_Record Dim filehandle As Integer Dim filename As String

Dim
> numLines as long Dim line As Long filehandle = FreeFile

filename =
> "E:\source.txt" Open filename For Random Access Read Lock Read

Write As
> #filehandle Len = Len(record) numLines = LOF(1) / Len(record)

For line
> = 2 to numLines Get #filehandle, line, record With

record
> query = query & CLng(.DUNS) & "," query = query & "'"

&
> Replace(Trim(.CO_NAME), "'", "''") & "'," query = query &

"'" &
> ascii_easy(.CO_NAME) & "'," query = query & "'" &

Trim(.PCODE) & "',"
> query = query & phone(.TEL_NBR) & "," If
> Len(Trim(.EMPS_COMP)) Then query = query & CLng(.EMPS_COMP) Else

query = query
> & "NULL" query = query & "," If Len(Trim(.LE))

Then query
> = query & CLng(.LE) Else query = query & "NULL" query =

query & ","
> query = query & CLng(.US72) & "," query = query &


> rs!source_id End With 'DELIVER THE DATA IN query TO THE

DRIVER
> Next lineI have tried several methods to deliver the prepared data

to the
> driver but without any succes.
> Writing to STDOUT
> Private Declare Function GetStdHandle Lib "Kernel32" (ByVal

nStdHandle
> As Long) As Long Private Declare Function WriteFile Lib "Kernel32"

(ByVal
> hFile As Long, ByVal lpBuffer As St

ring, ByVal nNumberOfBytesToWrite
As Long,
> lpNumberOfBytesWritten As Long, lpOverlapped As Any) As Long

Private Const
> STD_OUTPUT_HANDLE = -11& Dim stdhandle As Long Dim llResult As

Long
> stdhandle = GetStdHandle(STD_OUTPUT_HANDLE) WriteFile stdhandle,

query,
> Len(query), llResult, ByVal 0&Writing to a socket
> Dim socket As New Winsock With socket .Protocol =
> sckUDPProtocol .RemoteHost = "10.100.1.24" .RemotePort

= 2345
> .Connect End With socket.SendData queryExecuting it
> conn.Execute queryWriting to some stream
> Dim str As New Stream With str .Mode = adModeWrite


> .Open End With str.WriteText query
> So basically my question is : how do I deliver the prepared data to

the
> driver? Any help (tips, working code, example, ...) would be

appreciated.
> Best regards

Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn
Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben,
informieren Sie bitte den Absender und löschen Sie diese E-Mail. Das unerlaubte
Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht gestattet. Aus
Rechts- und Sicherheitsgründen ist die in dieser E-Mail gegebene Information nicht
rechtsverbindlich.

This e-mail contains confidential and/or privileged information. If you are not the
intended addressee or have received this e-mail in error please notify the sender and
delete this e-mail. Any unauthorized copying, disclosure or distribution of the material
in this e-mail is strictly forbidden. Due to legal and security reasons the information
contained in this e-mail is not legally binding.



Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 02:52 AM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0
www.UnixAdminTalk.com