Electronics & Programming

develissimo

Open Source electronics development and programming

  • You are not logged in.

#1 June 19, 2010 17:21:36

z.
Registered: 2009-11-02
Reputation: +  0  -
Profile   Send e-mail  

Django Sphinx or haystack Xapian


I need to implement search solution over models in mysql.

Anyone has experience with both or has gone through same decision
making?

What should I consider before choosing one or other?

(Mysql text search will not work as I use INNODB. Solr is powerful but
I heard it is very memory hungry and it is in Java. Whoosh is not yet
as mature as others. So that leaves choice between Django Sphinx and
Haystack Xapian.)

thanks


--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group athttp://groups.google.com/group/django-users?hl=en.

Offline

#2 June 19, 2010 18:50:31

Nick A.
Registered: 2009-11-02
Reputation: +  0  -
Profile   Send e-mail  

Django Sphinx or haystack Xapian


On Sat, Jun 19, 2010 at 9:21 AM, zweb <traderash...@gmail.com> wrote:

> I need to implement search solution over models in mysql.
>
> Anyone has experience with both or has gone through same decision
> making?
>
> What should I consider before choosing one or other?


I'm making a similar decision... as far as I can see, neither one does
search result clustering, which I'd like. But the memory required to host
Solr means a big increase in hosting costs. I'm using Webfaction, so to go
from their basic service to the one that offers 300 MB of memory is an
increase of about $25/month, which I'd rather not have to pay yet.

Nick

--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group athttp://groups.google.com/group/django-users?hl=en.

Offline

#3 June 19, 2010 18:54:41

Nick A.
Registered: 2009-11-02
Reputation: +  0  -
Profile   Send e-mail  

Django Sphinx or haystack Xapian


On Sat, Jun 19, 2010 at 9:21 AM, zweb <traderash...@gmail.com> wrote:

>
> (Mysql text search will not work as I use INNODB. Solr is powerful but
> I heard it is very memory hungry and it is in Java. Whoosh is not yet
> as mature as others. So that leaves choice between Django Sphinx and
> Haystack Xapian.)


I meant to mention that although I'm using InnoDB, I'm storing a copy of the
searchable text in a MyISAM table and using the MySQL search on it. That's
okay for now, but I'm looking for clustering, faceted search and other fancy
stuff.

I've worked in search-related technology for a long time and I should know
that search performance always demands lots of memory... So I may bite the
bullet and use Solr, after all. I just wish there were a way to trade off
the memory for speed until I'm ready to deploy a real working version.

Nick

--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group athttp://groups.google.com/group/django-users?hl=en.

Offline

#4 June 19, 2010 21:30:57

z.
Registered: 2009-11-02
Reputation: +  0  -
Profile   Send e-mail  

Django Sphinx or haystack Xapian


I am tilting towards Haystack Xapian solution over django sphinx
mainly for two reasons
1) Haystack supports - Solr, Xapian and Whoosh. So in future I can
easily migrate from Xapian to Solr as my need grows.
2) Sphinx has slow index update. Updating index takes as much time
as building a new one.


My views are based on quick reading of material and opinions on web
and not on my experience yet. Once I will try haystack xapian this
weekend and let you know.



On Jun 19, 10:54 am, Nick Arnett <nick.arn...@gmail.com> wrote:
> On Sat, Jun 19, 2010 at 9:21 AM, zweb <traderash...@gmail.com> wrote:
>
> > (Mysql text search will not work as I use INNODB. Solr is powerful but
> > I heard it is very memory hungry and it is in Java. Whoosh is not yet
> > as mature as others. So that leaves choice between Django Sphinx and
> > Haystack Xapian.)
>
> I meant to mention that although I'm using InnoDB, I'm storing a copy of the
> searchable text in a MyISAM table and using the MySQL search on it.  That's
> okay for now, but I'm looking for clustering, faceted search and other fancy
> stuff.
>
> I've worked in search-related technology for a long time and I should know
> that search performance always demands lots of memory...  So I may bite the
> bullet and use Solr, after all.  I just wish there were a way to trade off
> the memory for speed until I'm ready to deploy a real working version.
>
> Nick

--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group athttp://groups.google.com/group/django-users?hl=en.

Offline

#5 June 24, 2010 09:16:00

Dmitry D.
Registered: 2009-11-02
Reputation: +  0  -
Profile   Send e-mail  

Django Sphinx or haystack Xapian


Hi!

zweb wrote:
> 2) Sphinx has slow index update. Updating index takes as much time
> as building a new one.

I have a Sphinx indexed form with 2 million posts. Indexing takes mess than
2 minutes. Is that considered slow? I use full rebuild of the index even 10
minutes. Incremental reindexing is much faster but crashes from time to
time. Still I do not consider 2 minutes to be slow. There are much slower
insexing search engines around.

I chose Sphinx exactly because of its indexing and search speed. Docs say
that indexing is slow but I came from search engines that used hours to
index my forums. 2 minutes of Sphinx indexing is just nothing compared to
others.

--
Dmitry Dulepov
Twitter:http://twitter.com/dmitryd/Web:http://dmitry-dulepov.com/--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group athttp://groups.google.com/group/django-users?hl=en.

Offline

#6 June 24, 2010 12:04:05

e.
Registered: 2009-11-02
Reputation: +  0  -
Profile   Send e-mail  

Django Sphinx or haystack Xapian


We are in the process of switching from a custom Xapain installation
to Solr as we found Xapian quite limited in its ability to do
faceting, spelling suggestions, or highlighting. If you don't need any
of those things, I would recommend Xapian (although I've not used it
through Haystack). I have used Haystack with Whoosh and agree that
Whoosh needs a fair bit of work yet. That said I do like the haystack
idea.

Solr is really good for what we want and is really powerful, but we
have had to allocate it 1.5GB of memory for reindexing!

I have no experience of sphinx, but from what people have said above
it seems like a good solution.

Euan

On Jun 24, 9:15 am, Dmitry Dulepov <dmitry.dule...@gmail.com> wrote:
> Hi!
>
> zweb wrote:
> > 2) Sphinx  has slow index update.  Updating index takes as much time
> > as building a new one.
>
> I have a Sphinx indexed form with 2 million posts. Indexing takes mess than
> 2 minutes. Is that considered slow? I use full rebuild of the index even 10
> minutes. Incremental reindexing is much faster but crashes from time to
> time. Still I do not consider 2 minutes to be slow. There are much slower
> insexing search engines around.
>
> I chose Sphinx exactly because of its indexing and search speed. Docs say
> that indexing is slow but I came from search engines that used hours to
> index my forums. 2 minutes of Sphinx indexing is just nothing compared to
> others.
>
> --
> Dmitry Dulepov
> Twitter:http://twitter.com/dmitryd/> Web:http://dmitry-dulepov.com/--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group athttp://groups.google.com/group/django-users?hl=en.

Offline

Board footer

Moderator control

Enjoy the 15th of December
PoweredBy

The Forums are managed by develissimo stuff members, if you find any issues or misplaced content please help us to fix it. Thank you! Tell us via Contact Options
Leave a Message
Welcome to Develissimo Live Support