Hi all,

I have somebody creating a C# class to migrate data from one SQL database to
another slightly differently structured SQL database.

Please can anybody suggest ways to verrify that the data has migrated
successfully, in whole and acurrate??

I feel a sample approach would not quite be substancial.
I want to keep it seperate from the migration process itself (having my
person write a verification script may also not work as he will be using the
same thought processes and knowledge that he used for the migration)

Free Software, scripts, utilities, packages, industry approaches??

Sorry Im no Tech wizzard, Any ideas appreciated.....................
--
View this message in context: http://www.nabble.com/Data-migration...html#a10913962
Sent from the MySQL - General mailing list archive at Nabble.com.

Besides the sample approach, output data (a set you would output on a live
system anyway) from both db setups and see if you can get the same output
from both

Olaf


On 6/1/07 10:35 AM, "paulizaz" <paulizaz@hotmail.com> wrote:

>
> Hi all,
>
> I have somebody creating a C# class to migrate data from one SQL database to
> another slightly differently structured SQL database.
>
> Please can anybody suggest ways to verrify that the data has migrated
> successfully, in whole and acurrate??
>
> I feel a sample approach would not quite be substancial.
> I want to keep it seperate from the migration process itself (having my
> person write a verification script may also not work as he will be using the
> same thought processes and knowledge that he used for the migration)
>
> Free Software, scripts, utilities, packages, industry approaches??
>
> Sorry Im no Tech wizzard, Any ideas appreciated.....................

What do you mean by "same output" ?

I have too much data to go through and check if all the data is the same.
This is my problem. Sampling would speed this up, but I need something more
accurate.
All data is important.




Olaf Stein-2 wrote:
>
> Besides the sample approach, output data (a set you would output on a live
> system anyway) from both db setups and see if you can get the same output
> from both
>
> Olaf
>
>
> On 6/1/07 10:35 AM, "paulizaz" <paulizaz@hotmail.com> wrote:
>
>>
>> Hi all,
>>
>> I have somebody creating a C# class to migrate data from one SQL database
>> to
>> another slightly differently structured SQL database.
>>
>> Please can anybody suggest ways to verrify that the data has migrated
>> successfully, in whole and acurrate??
>>
>> I feel a sample approach would not quite be substancial.
>> I want to keep it seperate from the migration process itself (having my
>> person write a verification script may also not work as he will be using
>> the
>> same thought processes and knowledge that he used for the migration)
>>
>> Free Software, scripts, utilities, packages, industry approaches??
>>
>> Sorry Im no Tech wizzard, Any ideas appreciated.....................
>
>
> --
> MySQL General Mailing List
> For list archives: http://lists.mysql.com/mysql
> To unsubscribe: http://lists.mysql.com/mysql?unsub=lists@nabble.com
>
>
>

--
View this message in context: http://www.nabble.com/Data-migration...html#a10953205
Sent from the MySQL - General mailing list archive at Nabble.com.

On Mon, June 4, 2007 9:31, paulizaz said:
>
> What do you mean by "same output" ?
Can you write a program to access both databases and have it check to see
if the data matches. A lot depends on how the structure changed. If the
new database rows have a one to one correspondence to the original
database then 1st count the rows. If it passes that test compare the data
in the new row with the corresponding data in the old row.

Bill
> I have too much data to go through and check if all the data is the same.
> This is my problem. Sampling would speed this up, but I need something
> more
> accurate.
> All data is important.
>
>
>
>
> Olaf Stein-2 wrote:
>>
>> Besides the sample approach, output data (a set you would output on a
>> live
>> system anyway) from both db setups and see if you can get the same
>> output
>> from both
>>
>> Olaf
>>
>>
>> On 6/1/07 10:35 AM, "paulizaz" <paulizaz@hotmail.com> wrote:
>>
>>>
>>> Hi all,
>>>
>>> I have somebody creating a C# class to migrate data from one SQL
>>> database
>>> to
>>> another slightly differently structured SQL database.
>>>
>>> Please can anybody suggest ways to verrify that the data has migrated
>>> successfully, in whole and acurrate??
>>>
>>> I feel a sample approach would not quite be substancial.
>>> I want to keep it seperate from the migration process itself (having my
>>> person write a verification script may also not work as he will be
>>> using
>>> the
>>> same thought processes and knowledge that he used for the migration)
>>>
>>> Free Software, scripts, utilities, packages, industry approaches??
>>>
>>> Sorry Im no Tech wizzard, Any ideas appreciated.....................
>>
>>
>> --
>> MySQL General Mailing List
>> For list archives: http://lists.mysql.com/mysql
>> To unsubscribe: http://lists.mysql.com/mysql?unsub=lists@nabble.com
>>
>>
>>

I don't mean the whole thing.
Pick some output that your applications usually produce and see if you can
get the same results for both databases.

I am not saying that this is the only and best way, just in addition to the
mentioned sample approach.

If you want to know for sure you will have to write an application that
checks this for you, even if it runs for a while


On 6/4/07 12:31 PM, "paulizaz" <paulizaz@hotmail.com> wrote:

>
> What do you mean by "same output" ?
>
> I have too much data to go through and check if all the data is the same.
> This is my problem. Sampling would speed this up, but I need something more
> accurate.
> All data is important.
>
>
>
>
> Olaf Stein-2 wrote:
>>
>> Besides the sample approach, output data (a set you would output on a live
>> system anyway) from both db setups and see if you can get the same output
>> from both
>>
>> Olaf
>>
>>
>> On 6/1/07 10:35 AM, "paulizaz" <paulizaz@hotmail.com> wrote:
>>
>>>
>>> Hi all,
>>>
>>> I have somebody creating a C# class to migrate data from one SQL database
>>> to
>>> another slightly differently structured SQL database.
>>>
>>> Please can anybody suggest ways to verrify that the data has migrated
>>> successfully, in whole and acurrate??
>>>
>>> I feel a sample approach would not quite be substancial.
>>> I want to keep it seperate from the migration process itself (having my
>>> person write a verification script may also not work as he will be using
>>> the
>>> same thought processes and knowledge that he used for the migration)
>>>
>>> Free Software, scripts, utilities, packages, industry approaches??
>>>
>>> Sorry Im no Tech wizzard, Any ideas appreciated.....................
>>
>>
>> --
>> MySQL General Mailing List
>> For list archives: http://lists.mysql.com/mysql
>> To unsubscribe: http://lists.mysql.com/mysql?unsub=lists@nabble.com
>>
>>
>>

Hi !


paulizaz wrote:
> What do you mean by "same output" ?
>
> I have too much data to go through and check if all the data is the same.
> This is my problem. Sampling would speed this up, but I need something more
> accurate.
> All data is important.

In Unix / Linux, you would generate similar plain-text output before and
after the migration ("select *" to file, any report, ...) and then
compare them, typically using "diff".
If the format differs on intention (more/fewer columns, sequence of
columns, ...), you would include "sed" or some other filter to align the
two.
As a last resort, you would write some small script (Perl, shell, ...)
to compare them.

For me, it is a very common task to take the output of two different
versions of something (a build log, a trace from a run, the log of a
backup run, ...) and compare them via such a tool chain.


As you mentioned C#, I take it you are on Windows - pick your equivalent
available.


HTH,
Joerg

--
Joerg Bruehe, Senior Production Engineer
MySQL AB, www.mysql.com