Login ProductsSalesSupportDownloadsAbout |
Home » Technical Support » DBISAM Technical Support » Support Forums » DBISAM General » View Thread |
Messages 1 to 2 of 2 total |
OnTextIndexFilter performance (lack of) |
Wed, May 3 2006 8:00 AM | Permanent Link |
Roy Lambert NLH Associates Team Elevate | XPsp2, D6sp2, DBISAM 4.22b5
I'm rewriting an app and decided for the email part that rather than decode emails on arrival and store the bits in separate fields (_Headers, _Message, _InLinPics, _Attachments) I'd just receive them and store in encoded format (splitting out subject etc for speed of display in a grid) and decode as and when necessary. The problem I'm encountering is with full text indexing on the message. Decode the message, strip to plain text, massage to leaving just the words I want. If I create an extra field (_FastSearch), set up full text indexing and then loop through the table decoding the messages and stuffing the things I want indexing into _FastSearch its fine. The average time taken per record is Decode and stuff into _FastSearch - 11ms Decode and stuff into _FastSearch with fti - 35ms As you can see fti gives a fair performance hit - but still very acceptable. However, if I try and use the same code in OnTextIndexFilter and use sql to add the encoded field to the full text index I don't know how long things take because I decided to stop it after 12 hours. I don't want to store the data multiple times if I can avoid it - any suggestions. In ELN there are about 25k records in total, 1.5k records where there is no data in _Encoded (the field holding the message data), about 15k that are plain text, less than 1k are rtf and the remainder are mime. Roy Lambert Attachments: DialToMdi1.zip |
Wed, May 3 2006 11:07 AM | Permanent Link |
Roy Lambert NLH Associates Team Elevate | I feel like a complete prat - I had a test in (FileName <> '_encoded' ) should have been a capital E
Poor old fti system was trying to index the mime encoded stuff including upto 15Mb of attachments so its forgiven for taking so long Roy Lambert |
This web page was last updated on Tuesday, September 17, 2024 at 04:19 AM | Privacy PolicySite Map © 2024 Elevate Software, Inc. All Rights Reserved Questions or comments ? E-mail us at info@elevatesoft.com |