vBulletin Search Engine Optimization
| |||||||
| Register | FAQ | Members List | Calendar | Search | Today's Posts | Mark Forums Read |
| ||||
| Hi. The main table for our site is called properties and it gets hit quite often (several times per second) something like: Queries Total: 41,496 Avg/Sec: 6.89 Slow: 0 Cache Hits : 15,096 Avg/Sec: 2.51 Now/Sec: 0.00 Ratio: 36.38% Threads Total: 1 Active: 1 Cached: 76 Key Efficiency: 94.41% Bytes in: 114 Bytes out: 6,713 This properties table is very simple. (Pasted below) There is about 500,000 rows in the table and we are experiencing long queries like: SELECT * FROM properties WHERE 1 =1 AND properties.Published <>0 AND properties.Deleted <>1 AND properties.state = 'ca' AND TYPE = 'Residential' AND Image1 <> '' ORDER BY id DESC LIMIT 0 , 35 An explain on that yields: | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+------------+-------------+-----------------------------+-----------------+---------+------+-------+---------------------------------------------------------------+ | 1 | SIMPLE | properties | index_merge | Type,TypeSubType,StateIndex | Type,StateIndex | 1,67 | NULL | 45048 | Using intersect(Type,StateIndex); Using where; Using filesort | Is there anything you can see with the table or key design that might be causing this slowdown? There are 5 databases: 1 master, 4 slaves replicated. The master is only used for INSERTs, UPDATEs and DELETEs. The properties table is INNODB. Should it me MyISAM? Thanks for any help! Michael CREATE TABLE properties ( id int(11) unsigned NOT NULL auto_increment, UserID int(11) unsigned NOT NULL default '0', `Type` enum('Commercial','Residential') NOT NULL default 'Residential', Subtype varchar(64) NOT NULL default '0', Zip varchar(10) default '', Heading varchar(84) NOT NULL default '', Address1 varchar(128) NOT NULL default '', Address2 varchar(32) default NULL, Unit varchar(32) default NULL, Neighborhood varchar(64) default NULL, City varchar(64) NOT NULL default '0', State varchar(64) default '', Country varchar(4) default 'USA', ..... ....... ....... ListingContactHTML varchar(255) default NULL, IsShare tinyint(1) default '0', IsSublet tinyint(1) default '0', PRIMARY KEY (id), KEY `Type` (`Type`), KEY Subtype (Subtype), KEY TypeSubType (`Type`,Subtype), KEY CityHood (City,Neighborhood), KEY GoogleBase (GoogleBase), KEY Zip (Zip), KEY AddressSearch (Heading,Zip,City,Neighborhood,Address1,Unit), KEY StateIndex (State), KEY ListingContactRemoteCode (ListingContactRemoteCode), KEY LeaseType (LeaseType), KEY CreationDate (CreationDate), KEY LastMapLookup (LastMapLookup), KEY UserID (UserID), KEY Country (Country), KEY LatLon (lat,lon), KEY CityStateType (City,State,`Type`), KEY BatchUpdateRemoteListingID (BatchUpdateRemoteListingID), KEY CountryType (Country,`Type`), KEY Country_2 (Country,City,State) ) ENGINE=InnoDB AUTO_INCREMENT=907758 DEFAULT CHARSET=latin1 AUTO_INCREMENT=907758 ; |
| |||
| As a note. The query itself may not be taking long but there are many "Sorting result " and "Copying to tmp table " in myTop. Thanks, Michael On Tue, May 6, 2008 at 3:26 PM, Michael Stearne <mstearne@entermix.com> wrote: > Hi. > > The main table for our site is called properties and it gets hit quite > often (several times per second) something like: > > Queries Total: 41,496 Avg/Sec: 6.89 Slow: 0 > Cache Hits : 15,096 Avg/Sec: 2.51 Now/Sec: 0.00 Ratio: 36.38% > Threads Total: 1 Active: 1 Cached: 76 > Key Efficiency: 94.41% Bytes in: 114 Bytes out: 6,713 > > This properties table is very simple. (Pasted below) There is about > 500,000 rows in the table and we are experiencing long queries like: > > SELECT * FROM properties WHERE 1 =1 AND properties.Published <>0 AND > properties.Deleted <>1 AND properties.state = 'ca' AND TYPE = > 'Residential' AND Image1 <> '' ORDER BY id DESC LIMIT 0 , 35 > > An explain on that yields: > > | id | select_type | table | type | possible_keys > | key | key_len | ref | rows | Extra > | > +----+-------------+------------+-------------+-----------------------------+-----------------+---------+------+-------+---------------------------------------------------------------+ > | 1 | SIMPLE | properties | index_merge | > Type,TypeSubType,StateIndex | Type,StateIndex | 1,67 | NULL | 45048 > | Using intersect(Type,StateIndex); Using where; Using filesort | > > Is there anything you can see with the table or key design that might > be causing this slowdown? There are 5 databases: 1 master, 4 slaves > replicated. The master is only used for INSERTs, UPDATEs and DELETEs. > The properties table is INNODB. Should it me MyISAM? > > Thanks for any help! > Michael > > > CREATE TABLE properties ( > id int(11) unsigned NOT NULL auto_increment, > UserID int(11) unsigned NOT NULL default '0', > `Type` enum('Commercial','Residential') NOT NULL default 'Residential', > Subtype varchar(64) NOT NULL default '0', > Zip varchar(10) default '', > Heading varchar(84) NOT NULL default '', > Address1 varchar(128) NOT NULL default '', > Address2 varchar(32) default NULL, > Unit varchar(32) default NULL, > Neighborhood varchar(64) default NULL, > City varchar(64) NOT NULL default '0', > State varchar(64) default '', > Country varchar(4) default 'USA', > ..... > ...... > ...... > ListingContactHTML varchar(255) default NULL, > IsShare tinyint(1) default '0', > IsSublet tinyint(1) default '0', > PRIMARY KEY (id), > KEY `Type` (`Type`), > KEY Subtype (Subtype), > KEY TypeSubType (`Type`,Subtype), > KEY CityHood (City,Neighborhood), > KEY GoogleBase (GoogleBase), > KEY Zip (Zip), > KEY AddressSearch (Heading,Zip,City,Neighborhood,Address1,Unit), > KEY StateIndex (State), > KEY ListingContactRemoteCode (ListingContactRemoteCode), > KEY LeaseType (LeaseType), > KEY CreationDate (CreationDate), > KEY LastMapLookup (LastMapLookup), > KEY UserID (UserID), > KEY Country (Country), > KEY LatLon (lat,lon), > KEY CityStateType (City,State,`Type`), > KEY BatchUpdateRemoteListingID (BatchUpdateRemoteListingID), > KEY CountryType (Country,`Type`), > KEY Country_2 (Country,City,State) > ) ENGINE=InnoDB AUTO_INCREMENT=907758 DEFAULT CHARSET=latin1 > AUTO_INCREMENT=907758 ; > |
| |||
| Hi The query is not optimized as it is scanning 45048 rows. Vertical partitioning can be used because there is a lot of column in single table. On Tue, May 6, 2008 at 3:26 PM, Michael Stearne <mstearne@entermix.com> wrote: > Hi. > > The main table for our site is called properties and it gets hit quite > often (several times per second) something like: > > Queries Total: 41,496 Avg/Sec: 6.89 Slow: 0 > Cache Hits : 15,096 Avg/Sec: 2.51 Now/Sec: 0.00 Ratio: 36.38% > Threads Total: 1 Active: 1 Cached: 76 > Key Efficiency: 94.41% Bytes in: 114 Bytes out: 6,713 > > This properties table is very simple. (Pasted below) There is about > 500,000 rows in the table and we are experiencing long queries like: > > SELECT * FROM properties WHERE 1 =1 AND properties.Published <>0 AND > properties.Deleted <>1 AND properties.state = 'ca' AND TYPE = > 'Residential' AND Image1 <> '' ORDER BY id DESC LIMIT 0 , 35 > > An explain on that yields: > > | id | select_type | table | type | possible_keys > | key | key_len | ref | rows | Extra > | > > +----+-------------+------------+-------------+-----------------------------+-----------------+---------+------+-------+---------------------------------------------------------------+ > | 1 | SIMPLE | properties | index_merge | > Type,TypeSubType,StateIndex | Type,StateIndex | 1,67 | NULL | 45048 > | Using intersect(Type,StateIndex); Using where; Using filesort | > > Is there anything you can see with the table or key design that might > be causing this slowdown? There are 5 databases: 1 master, 4 slaves > replicated. The master is only used for INSERTs, UPDATEs and DELETEs. > The properties table is INNODB. Should it me MyISAM? > > Thanks for any help! > Michael > > > CREATE TABLE properties ( > id int(11) unsigned NOT NULL auto_increment, > UserID int(11) unsigned NOT NULL default '0', > `Type` enum('Commercial','Residential') NOT NULL default 'Residential', > Subtype varchar(64) NOT NULL default '0', > Zip varchar(10) default '', > Heading varchar(84) NOT NULL default '', > Address1 varchar(128) NOT NULL default '', > Address2 varchar(32) default NULL, > Unit varchar(32) default NULL, > Neighborhood varchar(64) default NULL, > City varchar(64) NOT NULL default '0', > State varchar(64) default '', > Country varchar(4) default 'USA', > ..... > ...... > ...... > ListingContactHTML varchar(255) default NULL, > IsShare tinyint(1) default '0', > IsSublet tinyint(1) default '0', > PRIMARY KEY (id), > KEY `Type` (`Type`), > KEY Subtype (Subtype), > KEY TypeSubType (`Type`,Subtype), > KEY CityHood (City,Neighborhood), > KEY GoogleBase (GoogleBase), > KEY Zip (Zip), > KEY AddressSearch (Heading,Zip,City,Neighborhood,Address1,Unit), > KEY StateIndex (State), > KEY ListingContactRemoteCode (ListingContactRemoteCode), > KEY LeaseType (LeaseType), > KEY CreationDate (CreationDate), > KEY LastMapLookup (LastMapLookup), > KEY UserID (UserID), > KEY Country (Country), > KEY LatLon (lat,lon), > KEY CityStateType (City,State,`Type`), > KEY BatchUpdateRemoteListingID (BatchUpdateRemoteListingID), > KEY CountryType (Country,`Type`), > KEY Country_2 (Country,City,State) > ) ENGINE=InnoDB AUTO_INCREMENT=907758 DEFAULT CHARSET=latin1 > AUTO_INCREMENT=907758 ; > > -- > MySQL General Mailing List > For list archives: http://lists.mysql.com/mysql > To unsubscribe: > http://lists.mysql.com/mysql?unsub=p...tikc@gmail.com > > -- Krishna Chandra Prajapati MySQL DBA, Ed Ventures e-Learning Pvt.Ltd. 1-8-303/48/15, Sindhi Colony P.G.Road, Secunderabad. Pin Code: 500003 Office Number: 040-66489771 Mob: 9912924044 URL: ed-ventures-online.com Email-id: prajapatikc@gmail.com |
| |||
| Hey all - I have two tables - an event_log table, and a user table. There is a "last_visit" column in the user table, and I want to update it from the event_log with the most recent event timestamp. And I want to do it without a subquery, eventually, both these tables will be pretty large, especially the event_log. I tried this: update enduser E join event_log EL on EL.enduser_acnt = E.enduser_acnt set E.last_visit = MAX(EL.event_time) group by EL.enduser_acnt but I get an error on the group by. The pertinent tables sections are: table event_log event_time TIMESTAMP enduser_acnt int table enduser enduser_acnt int last_visit datetime Any help appreciated. Thanks... andy -- Andy Wallace - CISData - IDX Slave AIM: acmwallace awallace@cisdata.net |
| |||
| Clarification: I DON'T want to update the last_visit field if there is no matching event record... I managed to get this to sort of work: update enduser E set E.last_visit = (select MAX(EL.event_time) from event_log EL where EL.enduser_acnt = E.enduser_acnt group by EL.enduser_acnt); but it updated the last_visit field to the default value if it found no matching event_log row... which I don't want to happen. thanks, andy Andy Wallace wrote: > Hey all - > I have two tables - an event_log table, and a user table. There is > a "last_visit" column in the user table, and I want to update it from > the event_log with the most recent event timestamp. And I want to do > it without a subquery, eventually, both these tables will be pretty > large, especially the event_log. > > I tried this: > > update enduser E join event_log EL on EL.enduser_acnt = E.enduser_acnt > set E.last_visit = MAX(EL.event_time) > group by EL.enduser_acnt > > but I get an error on the group by. The pertinent tables sections are: > > table event_log > event_time TIMESTAMP > enduser_acnt int > > table enduser > enduser_acnt int > last_visit datetime > > Any help appreciated. Thanks... > andy > > -- Andy Wallace - CISData - IDX Slave AIM: acmwallace awallace@cisdata.net |
| |||
| Hi All, I've the following query :SELECT ProductID FROM Products WHERE Enabled= 'Yes' AND ProductID IN(varProductID) This query works fine. However the query result is in a different order towhat I passed in varProductID. How can I order the results based on my list like varProductID = "1000,2500,1500" At the moment the result is 1000 1500 2500 But I want 1000 2500 1500 Thanks, Neil __________________________________________________ _______________ Discover and Win with Live Search http://clk.atdmt.com/UKM/go/msnnkmgl...ukm/direct/01/ |
| |||
| I want to put only the max date into the field... I was thinking that max was a group function, but now that I type that out loud, perhaps I'm not using all the neurons available... hmmm... thanks, andy Martin wrote: > Hi Andy- > > Is there a reason why you are using Query group by clause in UPDATE > statement? > > M > ----- Original Message ----- From: "Andy Wallace" <awallace@cisdata.net> > To: "mysql list" <mysql@lists.mysql.com> > Sent: Wednesday, May 07, 2008 1:07 PM > Subject: Re: question about update/join query > > >> Clarification: I DON'T want to update the last_visit field if there >> is no matching event record... >> >> I managed to get this to sort of work: >> >> update enduser E >> set E.last_visit = (select MAX(EL.event_time) >> from event_log EL >> where EL.enduser_acnt = E.enduser_acnt >> group by EL.enduser_acnt); >> >> but it updated the last_visit field to the default value if it found >> no matching event_log row... which I don't want to happen. >> >> thanks, >> andy >> >> Andy Wallace wrote: >>> Hey all - >>> I have two tables - an event_log table, and a user table. There is >>> a "last_visit" column in the user table, and I want to update it from >>> the event_log with the most recent event timestamp. And I want to do >>> it without a subquery, eventually, both these tables will be pretty >>> large, especially the event_log. >>> >>> I tried this: >>> >>> update enduser E join event_log EL on EL.enduser_acnt = E.enduser_acnt >>> set E.last_visit = MAX(EL.event_time) >>> group by EL.enduser_acnt >>> >>> but I get an error on the group by. The pertinent tables sections are: >>> >>> table event_log >>> event_time TIMESTAMP >>> enduser_acnt int >>> >>> table enduser >>> enduser_acnt int >>> last_visit datetime >>> >>> Any help appreciated. Thanks... >>> andy >>> >>> >> >> -- >> Andy Wallace - CISData - IDX Slave >> AIM: acmwallace awallace@cisdata.net >> >> -- >> MySQL General Mailing List >> For list archives: http://lists.mysql.com/mysql >> To unsubscribe: http://lists.mysql.com/mysql?unsub=mgainty@hotmail.com >> >> > -- Andy Wallace - CISData - IDX Slave AIM: acmwallace awallace@cisdata.net |
| |||
| Ok, I think I need to try to restate my problem. I have an event_log table, which tracks events (!). Basic structure is: table: event_log event_time timestamp event_id int user_id int and my user table: table user user_id int name varchar(50) last_visit datetime I want to run a query that updates the last_visit column of user with the MAX(event_time) row for which the user_id's match, but only if I find an event: update user U set U.last_visit = (select max(L.event_time) from event_log L where L.user_id = U.user_id I would like to do it without a subquery, I thought that the multiple table syntax for UPDATE would do it, but I can't wrap my head around it. UPDATE user U, event_log L SET U.last_visit = MAX(L.event_time) WHERE U.user_id = L.user_id GROUP BY L.event_time I guess the main question is - CAN I do this? Or will I have to resort to either a subquery, or external processing? thanks, andy Martin wrote: > Hi Andy- > > the MAX function needs group by for the column for which it calculating > max value as in this example > (select MAX(EL.event_time) > // from event_log EL > // where EL.enduser_acnt = E.enduser_acnt > > //Inner join forces selection on columns which contain non null values > as seen here > from event_log AS EL INNER JOIN Event AS E > ON EL.enduser_acnt = Event.enduser_anct > > group by EL.event_time); > // group by EL.enduser_acnt); > > HTH > Martin > ----- Original Message ----- From: "Andy Wallace" <awallace@cisdata.net> > To: "Martin" <mgainty@hotmail.com> > Cc: "mysql list" <mysql@lists.mysql.com> > Sent: Wednesday, May 07, 2008 6:21 PM > Subject: Re: question about update/join query > > >> I want to put only the max date into the field... I was thinking that max >> was a group function, but now that I type that out loud, perhaps I'm not >> using all the neurons available... hmmm... >> >> >> thanks, >> andy >> >> Martin wrote: >>> Hi Andy- >>> >>> Is there a reason why you are using Query group by clause in UPDATE >>> statement? >>> >>> M >>> ----- Original Message ----- From: "Andy Wallace" <awallace@cisdata.net> >>> To: "mysql list" <mysql@lists.mysql.com> >>> Sent: Wednesday, May 07, 2008 1:07 PM >>> Subject: Re: question about update/join query >>> >>> >>>> Clarification: I DON'T want to update the last_visit field if there >>>> is no matching event record... >>>> >>>> I managed to get this to sort of work: >>>> >>>> update enduser E >>>> set E.last_visit = (select MAX(EL.event_time) >>>> from event_log EL >>>> where EL.enduser_acnt = E.enduser_acnt >>>> group by EL.enduser_acnt); >>>> >>>> but it updated the last_visit field to the default value if it found >>>> no matching event_log row... which I don't want to happen. >>>> >>>> thanks, >>>> andy >>>> >>>> Andy Wallace wrote: >>>>> Hey all - >>>>> I have two tables - an event_log table, and a user table. There is >>>>> a "last_visit" column in the user table, and I want to update it from >>>>> the event_log with the most recent event timestamp. And I want to do >>>>> it without a subquery, eventually, both these tables will be pretty >>>>> large, especially the event_log. >>>>> >>>>> I tried this: >>>>> >>>>> update enduser E join event_log EL on EL.enduser_acnt = E.enduser_acnt >>>>> set E.last_visit = MAX(EL.event_time) >>>>> group by EL.enduser_acnt >>>>> >>>>> but I get an error on the group by. The pertinent tables sections are: >>>>> >>>>> table event_log >>>>> event_time TIMESTAMP >>>>> enduser_acnt int >>>>> >>>>> table enduser >>>>> enduser_acnt int >>>>> last_visit datetime >>>>> >>>>> Any help appreciated. Thanks... >>>>> andy >>>>> >>>>> >>>> >>>> -- >>>> Andy Wallace - CISData - IDX Slave >>>> AIM: acmwallace awallace@cisdata.net >>>> >>>> -- >>>> MySQL General Mailing List >>>> For list archives: http://lists.mysql.com/mysql >>>> To unsubscribe: http://lists.mysql.com/mysql?unsub=mgainty@hotmail.com >>>> >>>> >>> >> >> -- >> Andy Wallace - CISData - IDX Slave >> AIM: acmwallace awallace@cisdata.net >> > -- Andy Wallace - CISData - IDX Slave AIM: acmwallace awallace@cisdata.net |
| |||
| Ok, I think I need to try to restate my problem. I have an event_log table, which tracks events (!). Basic structure is: table: event_log event_time timestamp event_id int user_id int and my user table: table: user user_id int name varchar(50) last_visit datetime I want to run a query that updates the last_visit column of user with the MAX(event_time) row for which the user_id's match, but only if I find an event: update user U set U.last_visit = (select max(L.event_time) from event_log L where L.user_id = U.user_id) I would like to do it without a subquery, I thought that the multiple table syntax for UPDATE would do it, but I can't wrap my head around it. UPDATE user U, event_log L SET U.last_visit = MAX(L.event_time) WHERE U.user_id = L.user_id GROUP BY L.event_time I guess the main question is - CAN I do this? Or will I have to resort to either a subquery, or external processing? thanks, andy Martin wrote: > Hi Andy- > > the MAX function needs group by for the column for which it calculating > max value as in this example > (select MAX(EL.event_time) > // from event_log EL > // where EL.enduser_acnt = E.enduser_acnt > > //Inner join forces selection on columns which contain non null values > as seen here > from event_log AS EL INNER JOIN Event AS E > ON EL.enduser_acnt = Event.enduser_anct > > group by EL.event_time); > // group by EL.enduser_acnt); > > HTH > Martin > ----- Original Message ----- From: "Andy Wallace" <awallace@cisdata.net> > To: "Martin" <mgainty@hotmail.com> > Cc: "mysql list" <mysql@lists.mysql.com> > Sent: Wednesday, May 07, 2008 6:21 PM > Subject: Re: question about update/join query > > >> I want to put only the max date into the field... I was thinking that max >> was a group function, but now that I type that out loud, perhaps I'm not >> using all the neurons available... hmmm... >> >> >> thanks, >> andy >> >> Martin wrote: >>> Hi Andy- >>> >>> Is there a reason why you are using Query group by clause in UPDATE >>> statement? >>> >>> M >>> ----- Original Message ----- From: "Andy Wallace" <awallace@cisdata.net> >>> To: "mysql list" <mysql@lists.mysql.com> >>> Sent: Wednesday, May 07, 2008 1:07 PM >>> Subject: Re: question about update/join query >>> >>> >>>> Clarification: I DON'T want to update the last_visit field if there >>>> is no matching event record... >>>> >>>> I managed to get this to sort of work: >>>> >>>> update enduser E >>>> set E.last_visit = (select MAX(EL.event_time) >>>> from event_log EL >>>> where EL.enduser_acnt = E.enduser_acnt >>>> group by EL.enduser_acnt); >>>> >>>> but it updated the last_visit field to the default value if it found >>>> no matching event_log row... which I don't want to happen. >>>> >>>> thanks, >>>> andy >>>> >>>> Andy Wallace wrote: >>>>> Hey all - >>>>> I have two tables - an event_log table, and a user table. There is >>>>> a "last_visit" column in the user table, and I want to update it from >>>>> the event_log with the most recent event timestamp. And I want to do >>>>> it without a subquery, eventually, both these tables will be pretty >>>>> large, especially the event_log. >>>>> >>>>> I tried this: >>>>> >>>>> update enduser E join event_log EL on EL.enduser_acnt = E.enduser_acnt >>>>> set E.last_visit = MAX(EL.event_time) >>>>> group by EL.enduser_acnt >>>>> >>>>> but I get an error on the group by. The pertinent tables sections are: >>>>> >>>>> table event_log >>>>> event_time TIMESTAMP >>>>> enduser_acnt int >>>>> >>>>> table enduser >>>>> enduser_acnt int >>>>> last_visit datetime >>>>> >>>>> Any help appreciated. Thanks... >>>>> andy >>>>> >>>>> >>>> >>>> -- >>>> Andy Wallace - CISData - IDX Slave >>>> AIM: acmwallace awallace@cisdata.net >>>> >>>> -- >>>> MySQL General Mailing List >>>> For list archives: http://lists.mysql.com/mysql >>>> To unsubscribe: http://lists.mysql.com/mysql?unsub=mgainty@hotmail.com >>>> >>>> >>> >> >> -- >> Andy Wallace - CISData - IDX Slave >> AIM: acmwallace awallace@cisdata.net >> > -- Andy Wallace - CISData - IDX Slave AIM: acmwallace awallace@cisdata.net |
| ||||
| Perfect. It worked just how I wanted. Thanks for your help. Neil > Date: Wed, 7 May 2008 19:54:39 +0200> To: neildtompkins@hotmail.com> Subject: Re: Order Problem> From: leannonn@gmail.com> > Hi,> > You should look at the `FIND_IN_SET` function here: > http://dev.mysql.com/doc/refman/5.0/en/string-functions.html#function_find-in-set> > Your query could look like:> SELECT ProductID FROM Products WHERE Enabled= ' Yes' AND ProductID > IN(varProductID) ORDER BY FIND_IN_SET(ProductID, varProductID);> > Haven't tested it, though...> > > Take care,> Aleksandar __________________________________________________ _______________ Discover and Win with Live Search http://clk.atdmt.com/UKM/go/msnnkmgl...ukm/direct/01/ |