Electronics & Programming

develissimo

Open Source electronics development and programming

  • You are not logged in.

#1 Dec. 17, 2010 09:28:48

Shamail T.
Registered: 2009-11-02
Reputation: +  0  -
Profile   Send e-mail  

Django as huge image host


Hi,

I have this situation..

- We need to serve static files(images), lots of them, probably 100s
of them per second.

Our website is in Django and we need to support something like this:

1. a URL like /xyz.jpg should open an image.
2. image should be accesibe via this path only because we associate
some data while an image is served, viz number of hits, bandwidth
consumed, hits per second etc.

Prospects:
1. create something like cdn.sitename.com and on a hit to /xyz.png we
do something like

def servefile ( request ):
# save hit data
return HttpResponseRedirect ( "http://cdn.sitname.com/images/identifier.jpg" )
# where identifier is some id for this image.

(problem here is, users may directly start using the cdn version
of URL and we'll never know whats going on)

2. directly serve image, something like

def servefile ( request ):
# save hit data
ofile = file ( os.path.join (settings.UPLOAD_DIR,
"identifier.jpg") )

wrapper = FileWrapper ( file(ofile) )
response = HttpResponse ( wrapper, content_type='image/jpg' )
response = 'inline; filename=' +
uiImage.filerealname
response = os.path.getsize(ofile)
return response

(problem here is:
1. caching (which can be overcomed by methods like Etag and some
other ones)
2. super slow. (this, on an Amazon EC2 normal instance cannot
past scale of 3-4images simultaneously :( ))


Please give me some good ways to do this.

Thanks and regards

--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group athttp://groups.google.com/group/django-users?hl=en.

Offline

#2 Dec. 17, 2010 09:54:06

Erik C.
Registered: 2009-11-02
Reputation: +  0  -
Profile   Send e-mail  

Django as huge image host


Den 17/12/2010 kl. 10.28 skrev Shamail Tayyab:

> I have this situation..
>
> - We need to serve static files(images), lots of them, probably 100s
> of them per second.
>
> Our website is in Django and we need to support something like this:
>
> 1. a URL like /xyz.jpg should open an image.
> 2. image should be accesibe via this path only because we associate
> some data while an image is served, viz number of hits, bandwidth
> consumed, hits per second etc.

I'm not sure this is even a task for Django. If your primary concern is
performance, try doing this directly in Apache or another web server via
rewrite rules and use the access logs to do the accounting. Apache is good at
access statistics.

If your primary concern is statistics, there is no way you can get exact
figures. There may be any number of caching mechanisms between you and the
client that you don't control so GET requests may never reach your server.
Anyway, at this level there's no such thing as a "cdn version" because you are
supposedly in control of the web server that does the mapping from a GET
request to a file on the filesystem.

Eriksmime.p7sDescription:S/MIME cryptographic signature

Attachments:
attachment smime.p7s (1.9 KB)

Offline

#3 Dec. 17, 2010 10:43:48

Michel T.
Registered: 2009-11-02
Reputation: +  0  -
Profile   Send e-mail  

Django as huge image host


Hi,

> I'm not sure this is even a task for Django. If your primary concern is
> performance, try doing this directly in Apache or another web server via
> rewrite rules and use the access logs to do the accounting. Apache is good at
> access statistics.

Does your CDN offers statistics? I use rackspace cloud files and I
know there is log files I can rely on. I didn't used it yet but maybe
it's a choice...

--
Michel Sabchukhttp://turbosys.com.br/--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group athttp://groups.google.com/group/django-users?hl=en.

Offline

#4 Dec. 17, 2010 10:53:59

Tom E.
Registered: 2009-11-02
Reputation: +  0  -
Profile   Send e-mail  

Django as huge image host


On Fri, Dec 17, 2010 at 9:28 AM, Shamail Tayyab <pleoma...@gmail.com> wrote:
> Hi,
>
>    I have this situation..
>
> - We need to serve static files(images), lots of them, probably 100s
> of them per second.
>

Google for X-SendFile


Cheers

Tom

--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group athttp://groups.google.com/group/django-users?hl=en.

Offline

#5 Dec. 17, 2010 11:07:26

Shamail T.
Registered: 2009-11-02
Reputation: +  0  -
Profile   Send e-mail  

Django as huge image host


> I'm not sure this is even a task for Django. If your primary concern is
> performance, try doing this directly in Apache or another web server via
> rewrite rules and use the access logs to do the accounting. Apache is good at
> access statistics.
>
> If your primary concern is statistics, there is no way you can get exact
> figures. There may be any number of caching mechanisms between you and the
> client that you don't control so GET requests may never reach your server.
> Anyway, at this level there's no such thing as a "cdn version" because you
> are supposedly in control of the web server that does the mapping from a GET
> request to a file on the filesystem.
>

Hi,

Yes, the site and other functionality is being powered by Django,
its hardly related to Django. True, we are in control of the server,
but we can use another virtual host for CDN or something. Yes, log
files can be a good idea, but not optimal for much of the stats. I'll
get to it as last resort. Thanks

--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group athttp://groups.google.com/group/django-users?hl=en.

Offline

#6 Dec. 17, 2010 11:10:27

Shamail T.
Registered: 2009-11-02
Reputation: +  0  -
Profile   Send e-mail  

Django as huge image host


> Does your CDN offers statistics? I use rackspace cloud files and I
> know there is log files I can rely on. I didn't used it yet but maybe
> it's a choice...

Hi,

No, we don't yet have a CDN, its a prospect, a costly prospect.
Thanks

--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group athttp://groups.google.com/group/django-users?hl=en.

Offline

#7 Dec. 17, 2010 11:11:16

Shamail T.
Registered: 2009-11-02
Reputation: +  0  -
Profile   Send e-mail  

Django as huge image host


> Google for X-SendFile
>
Hey! This one looks like exactly what I want... I'll give it a shot.

Thanks

--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group athttp://groups.google.com/group/django-users?hl=en.

Offline

Board footer

Moderator control

Enjoy the 24th of October
PoweredBy

The Forums are managed by develissimo stuff members, if you find any issues or misplaced content please help us to fix it. Thank you! Tell us via Contact Options
Leave a Message
Welcome to Develissimo Live Support