This is a discussion on version/source control in datawarehouse project within the SQL Server Data Warehousing forums, part of the Microsoft SQL Server category; --> Hi! I'm a freshman in a datawarehouse company. All my coworkers are on-site in user side,such as banks, and ...
| |||||||
| Register | FAQ | Members List | Calendar | Search | Today's Posts | Mark Forums Read |
| ||||
| Hi! I'm a freshman in a datawarehouse company. All my coworkers are on-site in user side,such as banks, and our development machine is also given by user. There is no server or desktop in our company. After I join this company,I find there's no version/source control in our company's projects. Because I used to work in a web-based system development company, I think it's VERY important to have code under version control. In my new company,there are 1~3 members in each project. Everyone of them will terminal to server,open the same project and modify it. I try to think if there is a better way to improve this develop cycle. First of all,I have to have a version control server,such as subversion. Because we don't have a server in our company ,maybe installing SVN on development server is a good way. When sb wants to modify the code(SSIS/SSRS), he has to update his working copy to the latest version and modify it. But for me,there is a problem take SSIS for instance, how do we design datasource connectiion to make it available for every developer? if datasource is DB connection,I can change "localhost" to servername or ip to ensure connections on everyone's computer works. but for development server,would using servername instead of "localhost" make db connection lower?because it has to parse the servername? this is my first convern. If datasource is file connection,there is a similar situation,using relative path could make the code fit everyone's working environment but maybe inefficient when reading file? In the begining,I think maybe we can copy DB or files to developer's machine, but I find DB and files are too large(too much data...) And there are many abosolute path in our code So if I want to bring version/source control into this project,it seems we have to modify those path. After ETL,we have SSAS,SSRS to do. I want to know if there is a better way or best practice to design the version control of the datawarehouse project? For me,it seems like a mess now... Need some advice Thanks a lot!! |
| |||
| first, you can manage your SSIS, SSAS ans SSRS files under a source control system like visual source safe very easily. so, there is no issue here. second, you can use configuration files to store your database connections. each station need this config file located on the disk and setup for the developper env. (the config file is an xml file which can be edited by each developper to reflect the right servers names) make sure you use a folder like c:\myproject\ssisconfig which is created exactly the same way on all desktops. (there is other options to store and share the connections, but this one is easy to implement) third, for the version of the databases... you have to create a good process to manage SQL script files to keep everybody updated correctly. for example, organize this into folders: /Version1 /Patch1.1 /Patch1.2 .... each folder will contains SQL scripts which modify the database and each developer as to apply the patches on his own environment. Visual Studio for Database Professional can do this for you. But I recommend that only 1 person will be in charge of managing the database and the changes. http://msdn2.microsoft.com/en-us/tea.../aa718807.aspx and finally... communicate! :-) "YesGoGoGo" <YesGoGoGo@gmail.com> wrote in message news:1187628671.026950.145670@z24g2000prh.googlegr oups.com... > Hi! > I'm a freshman in a datawarehouse company. > All my coworkers are on-site in user side,such as banks, > and our development machine is also given by user. > There is no server or desktop in our company. > After I join this company,I find there's no version/source control in > our company's projects. > Because I used to work in a web-based system development company, > I think it's VERY important to have code under version control. > > In my new company,there are 1~3 members in each project. > Everyone of them will terminal to server,open the same project and > modify it. > I try to think if there is a better way to improve this develop cycle. > First of all,I have to have a version control server,such as > subversion. > Because we don't have a server in our company > ,maybe installing SVN on development server is a good way. > When sb wants to modify the code(SSIS/SSRS), > he has to update his working copy to the latest version and modify it. > > But for me,there is a problem > take SSIS for instance, how do we design datasource connectiion to > make it available for every developer? > if datasource is DB connection,I can change "localhost" to servername > or ip to ensure connections on everyone's computer works. > but for development server,would using servername instead of > "localhost" make db connection lower?because it has to parse the > servername? this is my first convern. > > If datasource is file connection,there is a similar situation,using > relative path could make the code fit everyone's working environment > but maybe inefficient when reading file? > > In the begining,I think maybe we can copy DB or files to developer's > machine, but I find DB and files are too large(too much data...) > > And there are many abosolute path in our code > So if I want to bring version/source control into this project,it > seems we have to modify those path. > > After ETL,we have SSAS,SSRS to do. > I want to know if there is a better way or best practice to design the > version control of the datawarehouse project? > For me,it seems like a mess now... > > Need some advice > Thanks a lot!! > |
| |||
| thanks to Jeje your advice is very useful for me! I've tried to build an environment like what you mention but there is still a problem about connection configuration. the following is my SSIS project file DTS_MIGRATE AC BACKUP DataSource DOWNLOAD DP FD LN OTHER I gather all datasources(*.ds) in DataSource folder and put *.dtsx into their corresponding folders So if the db connection on developer's machine is different from server's All he has to do is modify *.ds, right? But I've found in every *.dtsx db connection is also written in dtsx, like <DTS:Property DTS:Name="ConnectionString">Data Source=localhost;Initial Catalog=DW;Provider=SQLNCLI.1;Integrated Security=SSPI;</DTS:Property> that means developer has to modify all conn string in all dtsx.... that's so frustrating... Can't I just modify one place(*.ds) and make all dtsx work? there's another question about edting xml If I use ultraedit to modify *.ds db connection, although the xml has changed and saved After I open the project,these connecitons don't work I still have to modify these conn in IDE environment Maybe I'm not familiar to SSIS. still need help! thanks a lot! On 8 21 , 8 16 , "Jeje" <willg...@hotmail.com> wrote: > first, > you can manage your SSIS, SSAS ans SSRS files under a source control system > like visual source safe very easily. so, there is no issue here. > > second, > you can use configuration files to store your database connections. each > station need this config file located on the disk and setup for the > developper env. (the config file is an xml file which can be edited by each > developper to reflect the right servers names) > make sure you use a folder like c:\myproject\ssisconfig which is created > exactly the same way on all desktops. > (there is other options to store and share the connections, but this one is > easy to implement) > > third, > for the version of the databases... > you have to create a good process to manage SQL script files to keep > everybody updated correctly. for example, organize this into folders: > /Version1 > /Patch1.1 > /Patch1.2 > ... > each folder will contains SQL scripts which modify the database and each > developer as to apply the patches on his own environment. > Visual Studio for Database Professional can do this for you. > But I recommend that only 1 person will be in charge of managing the > database and the changes.http://msdn2.microsoft.com/en-us/tea.../aa718807.aspx > > and finally... communicate! :-) > > "YesGoGoGo" <YesGoG...@gmail.com> wrote in message > > news:1187628671.026950.145670@z24g2000prh.googlegr oups.com... > |
| ||||
| no, don't use DS files use configuration files (.dtsxconfig) right click in a package, then select the configuration option and you'll found a wizard to create these files for you. "YesGoGoGo" <YesGoGoGo@gmail.com> wrote in message news:1187839833.239317.268770@l22g2000prc.googlegr oups.com... > thanks to Jeje > your advice is very useful for me! > > I've tried to build an environment like what you mention > but there is still a problem about connection configuration. > > the following is my SSIS project file > DTS_MIGRATE > AC > BACKUP > DataSource > DOWNLOAD > DP > FD > LN > OTHER > I gather all datasources(*.ds) in DataSource folder and put *.dtsx > into their corresponding folders > So if the db connection on developer's machine is different from > server's > All he has to do is modify *.ds, right? > But I've found in every *.dtsx > db connection is also written in dtsx, like <DTS:Property > DTS:Name="ConnectionString">Data Source=localhost;Initial > Catalog=DW;Provider=SQLNCLI.1;Integrated Security=SSPI;</DTS:Property> > that means developer has to modify all conn string in all dtsx.... > that's so frustrating... > Can't I just modify one place(*.ds) and make all dtsx work? > > there's another question about edting xml > If I use ultraedit to modify *.ds db connection, although the xml has > changed and saved > After I open the project,these connecitons don't work > I still have to modify these conn in IDE environment > > Maybe I'm not familiar to SSIS. > > still need help! > thanks a lot! > > > On 8 21 , 8 16 , "Jeje" <willg...@hotmail.com> wrote: >> first, >> you can manage your SSIS, SSAS ans SSRS files under a source control >> system >> like visual source safe very easily. so, there is no issue here. >> >> second, >> you can use configuration files to store your database connections. each >> station need this config file located on the disk and setup for the >> developper env. (the config file is an xml file which can be edited by >> each >> developper to reflect the right servers names) >> make sure you use a folder like c:\myproject\ssisconfig which is created >> exactly the same way on all desktops. >> (there is other options to store and share the connections, but this one >> is >> easy to implement) >> >> third, >> for the version of the databases... >> you have to create a good process to manage SQL script files to keep >> everybody updated correctly. for example, organize this into folders: >> /Version1 >> /Patch1.1 >> /Patch1.2 >> ... >> each folder will contains SQL scripts which modify the database and each >> developer as to apply the patches on his own environment. >> Visual Studio for Database Professional can do this for you. >> But I recommend that only 1 person will be in charge of managing the >> database and the >> changes.http://msdn2.microsoft.com/en-us/tea.../aa718807.aspx >> >> and finally... communicate! :-) >> >> "YesGoGoGo" <YesGoG...@gmail.com> wrote in message >> >> news:1187628671.026950.145670@z24g2000prh.googlegr oups.com... >> > |
| Thread Tools | |
| Display Modes | |
|
|