g*g
2 楼
ehCache is probably what you are looking for.
【在 b******y 的大作中提到】
: 有人用么？据说facebook用得很猛，用来cache无数东西。
: 现在的系统里面有N多static的东西，不知道是不是也可以try一下
【在 b******y 的大作中提到】
: 有人用么？据说facebook用得很猛，用来cache无数东西。
: 现在的系统里面有N多static的东西，不知道是不是也可以try一下
k*r
3 楼
how does it compare with memcached for distributed cashes?
【在 g*****g 的大作中提到】
: ehCache is probably what you are looking for.
【在 g*****g 的大作中提到】
: ehCache is probably what you are looking for.
g*g
4 楼
Don't really have any experience with memcached.
But I believe, ehCache and other Java Cache solution (JBoss Cache
etc.) gives you more control and flexibility. e.g. You can configure
ehCache as Hibernate's secondary Cache. And it's not intrusive.
Also ehCache can run in your JVM. memcached is C based.
【在 k***r 的大作中提到】
: how does it compare with memcached for distributed cashes?
But I believe, ehCache and other Java Cache solution (JBoss Cache
etc.) gives you more control and flexibility. e.g. You can configure
ehCache as Hibernate's secondary Cache. And it's not intrusive.
Also ehCache can run in your JVM. memcached is C based.
【在 k***r 的大作中提到】
: how does it compare with memcached for distributed cashes?
k*r
5 楼
Right. ehcache has the advantage when it's a local cache.
memcached is a key-value pair store, better for mixed
(not only Java) solutions. I haven't got a chance to
try ehcache in a distributed way, yet.
【在 g*****g 的大作中提到】
: Don't really have any experience with memcached.
: But I believe, ehCache and other Java Cache solution (JBoss Cache
: etc.) gives you more control and flexibility. e.g. You can configure
: ehCache as Hibernate's secondary Cache. And it's not intrusive.
: Also ehCache can run in your JVM. memcached is C based.
memcached is a key-value pair store, better for mixed
(not only Java) solutions. I haven't got a chance to
try ehcache in a distributed way, yet.
【在 g*****g 的大作中提到】
: Don't really have any experience with memcached.
: But I believe, ehCache and other Java Cache solution (JBoss Cache
: etc.) gives you more control and flexibility. e.g. You can configure
: ehCache as Hibernate's secondary Cache. And it's not intrusive.
: Also ehCache can run in your JVM. memcached is C based.
b*y
6 楼
象这种cache solutions，能comfortable handle多大的cache?
象PDF template这种static data，如果不cache，
就是大IO(from File system)或者blob(from DB)，慢得很，
如果cahche，会不会导致频繁paging?
【在 k***r 的大作中提到】
: Right. ehcache has the advantage when it's a local cache.
: memcached is a key-value pair store, better for mixed
: (not only Java) solutions. I haven't got a chance to
: try ehcache in a distributed way, yet.
象PDF template这种static data，如果不cache，
就是大IO(from File system)或者blob(from DB)，慢得很，
如果cahche，会不会导致频繁paging?
【在 k***r 的大作中提到】
: Right. ehcache has the advantage when it's a local cache.
: memcached is a key-value pair store, better for mixed
: (not only Java) solutions. I haven't got a chance to
: try ehcache in a distributed way, yet.
k*r
7 楼
The OS has its OS level cache. It will try to fit
the disk space that's accessed the most in memory
as well. But the maximum is the memory on local box.
In distributed cache, you can multiply that by number
of boxes you have, and apply a simple hash to partition
the objects. So the answer is, it's only limited by
your hardware resources.
【在 b******y 的大作中提到】
: 象这种cache solutions，能comfortable handle多大的cache?
: 象PDF template这种static data，如果不cache，
: 就是大IO(from File system)或者blob(from DB)，慢得很，
: 如果cahche，会不会导致频繁paging?
the disk space that's accessed the most in memory
as well. But the maximum is the memory on local box.
In distributed cache, you can multiply that by number
of boxes you have, and apply a simple hash to partition
the objects. So the answer is, it's only limited by
your hardware resources.
【在 b******y 的大作中提到】
: 象这种cache solutions，能comfortable handle多大的cache?
: 象PDF template这种static data，如果不cache，
: 就是大IO(from File system)或者blob(from DB)，慢得很，
: 如果cahche，会不会导致频繁paging?
b*y
8 楼
i still don't have clear idea about the performance/capacity of
these cache solutions...
also, how can developers test product system performance?
dev env for individual developer can't have that much memory.
setting up a dev lab env for whole dev team?
【在 k***r 的大作中提到】
: The OS has its OS level cache. It will try to fit
: the disk space that's accessed the most in memory
: as well. But the maximum is the memory on local box.
: In distributed cache, you can multiply that by number
: of boxes you have, and apply a simple hash to partition
: the objects. So the answer is, it's only limited by
: your hardware resources.
these cache solutions...
also, how can developers test product system performance?
dev env for individual developer can't have that much memory.
setting up a dev lab env for whole dev team?
【在 k***r 的大作中提到】
: The OS has its OS level cache. It will try to fit
: the disk space that's accessed the most in memory
: as well. But the maximum is the memory on local box.
: In distributed cache, you can multiply that by number
: of boxes you have, and apply a simple hash to partition
: the objects. So the answer is, it's only limited by
: your hardware resources.
k*r
9 楼
Usually you can get a good idea how well it works by
turning on and off cache in a dev environment . You
can do it on one box with low relatively memory.
Your goal is to create a situation to make OS caching
not very effective. Then turn on memcached caching to
show the difference. The multi-box solution just scales
up the total amount of data.
【在 b******y 的大作中提到】
: i still don't have clear idea about the performance/capacity of
: these cache solutions...
: also, how can developers test product system performance?
: dev env for individual developer can't have that much memory.
: setting up a dev lab env for whole dev team?
turning on and off cache in a dev environment . You
can do it on one box with low relatively memory.
Your goal is to create a situation to make OS caching
not very effective. Then turn on memcached caching to
show the difference. The multi-box solution just scales
up the total amount of data.
【在 b******y 的大作中提到】
: i still don't have clear idea about the performance/capacity of
: these cache solutions...
: also, how can developers test product system performance?
: dev env for individual developer can't have that much memory.
: setting up a dev lab env for whole dev team?
m*t
10 楼
I don't think the goal should be comparing OS caching
and memcached (or any other caching solutions), because
for static files, you can't beat the OS, and for
non-static data, you can't use OS caching.
【在 k***r 的大作中提到】
: Usually you can get a good idea how well it works by
: turning on and off cache in a dev environment . You
: can do it on one box with low relatively memory.
: Your goal is to create a situation to make OS caching
: not very effective. Then turn on memcached caching to
: show the difference. The multi-box solution just scales
: up the total amount of data.
and memcached (or any other caching solutions), because
for static files, you can't beat the OS, and for
non-static data, you can't use OS caching.
【在 k***r 的大作中提到】
: Usually you can get a good idea how well it works by
: turning on and off cache in a dev environment . You
: can do it on one box with low relatively memory.
: Your goal is to create a situation to make OS caching
: not very effective. Then turn on memcached caching to
: show the difference. The multi-box solution just scales
: up the total amount of data.
k*r
11 楼
blogcity wants to cache files.
【在 m******t 的大作中提到】
: I don't think the goal should be comparing OS caching
: and memcached (or any other caching solutions), because
: for static files, you can't beat the OS, and for
: non-static data, you can't use OS caching.
【在 m******t 的大作中提到】
: I don't think the goal should be comparing OS caching
: and memcached (or any other caching solutions), because
: for static files, you can't beat the OS, and for
: non-static data, you can't use OS caching.
m*t
12 楼
In that case, like I said, I don't see how any 3rd party
cache solutions can be faster than OS.
【在 k***r 的大作中提到】
: blogcity wants to cache files.
g*g
13 楼
It's not faster, but not neccesarily slower. It's all about
putting files in memory and the eviction policy after all.
Third party solution may give you a pure java solution that
runs in the same JVM.
【在 m******t 的大作中提到】
:
: In that case, like I said, I don't see how any 3rd party
: cache solutions can be faster than OS.
putting files in memory and the eviction policy after all.
Third party solution may give you a pure java solution that
runs in the same JVM.
【在 m******t 的大作中提到】
:
: In that case, like I said, I don't see how any 3rd party
: cache solutions can be faster than OS.
k*r
14 楼
Several advantages over OS caching. You have control over
your policy of purging the cache. And you can go distributed
if you need to.
【在 m******t 的大作中提到】
:
: In that case, like I said, I don't see how any 3rd party
: cache solutions can be faster than OS.
your policy of purging the cache. And you can go distributed
if you need to.
【在 m******t 的大作中提到】
:
: In that case, like I said, I don't see how any 3rd party
: cache solutions can be faster than OS.
m*t
15 楼
You have got to be kidding me to believe that a pure java
based cache solution is "not necessarily slower" than
OS file caching.
【在 g*****g 的大作中提到】
: It's not faster, but not neccesarily slower. It's all about
: putting files in memory and the eviction policy after all.
: Third party solution may give you a pure java solution that
: runs in the same JVM.
m*t
16 楼
I thought of that. It's a fair point, but then remember we
are talking about static files, and the point I stressed earlier
that web servers are almost always dedicated boxes. So
it usually comes down to whether a particular file is frequently
requested or not. I just don't feel sophisticated cache policies
are really necessary for static files.
Caching static files distributedly? How is that any better
than a simple load balancer in front of a set of web servers,
each with the same files dep
【在 k***r 的大作中提到】
: Several advantages over OS caching. You have control over
: your policy of purging the cache. And you can go distributed
: if you need to.
k*r
17 楼
It depends on the usage. Sometimes some complications
come into play. For example, in some cases you want
to run the web service on one single box but the storage
on multiple boxes. When files are used/served, the web
service needs to massage the data, or need to authentica
the user, etc., etc..
【在 m******t 的大作中提到】
:
: I thought of that. It's a fair point, but then remember we
: are talking about static files, and the point I stressed earlier
: that web servers are almost always dedicated boxes. So
: it usually comes down to whether a particular file is frequently
: requested or not. I just don't feel sophisticated cache policies
: are really necessary for static files.
: Caching static files distributedly? How is that any better
: than a simple load balancer in front of a set of web servers,
: each with the same files dep
come into play. For example, in some cases you want
to run the web service on one single box but the storage
on multiple boxes. When files are used/served, the web
service needs to massage the data, or need to authentica
the user, etc., etc..
【在 m******t 的大作中提到】
:
: I thought of that. It's a fair point, but then remember we
: are talking about static files, and the point I stressed earlier
: that web servers are almost always dedicated boxes. So
: it usually comes down to whether a particular file is frequently
: requested or not. I just don't feel sophisticated cache policies
: are really necessary for static files.
: Caching static files distributedly? How is that any better
: than a simple load balancer in front of a set of web servers,
: each with the same files dep
s*n
18 楼
for facebook they don't worry about static contents, but the flood of
messages created by millions of users online at the same time, and users
want to see all new messages they care about in real time.
their memcached is pretty much a 30TB specialized in-memory database for all
things all live sessions need. from this point of view, the "real" database
on disk is just a backup device in case of disaster.
messages created by millions of users online at the same time, and users
want to see all new messages they care about in real time.
their memcached is pretty much a 30TB specialized in-memory database for all
things all live sessions need. from this point of view, the "real" database
on disk is just a backup device in case of disaster.
b*y
19 楼
okie... i gotta ask a newbie question here: how to invoke OS
caching capacity from my java code? how to control OS caching,
say, LRU parameters, force to expire, etc?
【在 m******t 的大作中提到】
:
: I thought of that. It's a fair point, but then remember we
: are talking about static files, and the point I stressed earlier
: that web servers are almost always dedicated boxes. So
: it usually comes down to whether a particular file is frequently
: requested or not. I just don't feel sophisticated cache policies
: are really necessary for static files.
: Caching static files distributedly? How is that any better
: than a simple load balancer in front of a set of web servers,
: each with the same files dep
caching capacity from my java code? how to control OS caching,
say, LRU parameters, force to expire, etc?
【在 m******t 的大作中提到】
:
: I thought of that. It's a fair point, but then remember we
: are talking about static files, and the point I stressed earlier
: that web servers are almost always dedicated boxes. So
: it usually comes down to whether a particular file is frequently
: requested or not. I just don't feel sophisticated cache policies
: are really necessary for static files.
: Caching static files distributedly? How is that any better
: than a simple load balancer in front of a set of web servers,
: each with the same files dep
m*t
20 楼
Do you need to do that from your java code? In fact, I'm not
sure serving static files from your java code is a good idea to
begin with. It takes a couple rounds of buffer copying before
the content of a file is even seen by the java code. Why do that
when you could simply serve them from a web server?
(Well access control would be the only reason I can think of.)
【在 b******y 的大作中提到】
: okie... i gotta ask a newbie question here: how to invoke OS
: caching capacity from my java code? how to control OS caching,
: say, LRU parameters, force to expire, etc?
m*t
21 楼
all
database
30TB in-memory db, huh? That's very impressive.
【在 s******n 的大作中提到】
: for facebook they don't worry about static contents, but the flood of
: messages created by millions of users online at the same time, and users
: want to see all new messages they care about in real time.
: their memcached is pretty much a 30TB specialized in-memory database for all
: things all live sessions need. from this point of view, the "real" database
: on disk is just a backup device in case of disaster.
g*g
22 楼
Flexibility and portablity, below was a real app I worked in ex-job.
Let's say you want to implement an web server
that can display emails, all emails are in MIME format so
you have to parse emails to get attachments in the first
place. And of course you want to cache them to avoid parsing
them again.
Now use a pure java solution, you have the control of the eviction
policy. You may give VIP members more cache space or longer
expiry time for example. Your caching is portable, config the
director
【在 m******t 的大作中提到】
:
: all
: database
: 30TB in-memory db, huh? That's very impressive.
Let's say you want to implement an web server
that can display emails, all emails are in MIME format so
you have to parse emails to get attachments in the first
place. And of course you want to cache them to avoid parsing
them again.
Now use a pure java solution, you have the control of the eviction
policy. You may give VIP members more cache space or longer
expiry time for example. Your caching is portable, config the
director
【在 m******t 的大作中提到】
:
: all
: database
: 30TB in-memory db, huh? That's very impressive.
b*y
23 楼
ft... I gave example le.
let's say I need to generate PDF file from some PDF template files.
then the PDF template files are the files my java code read frequently.
【在 m******t 的大作中提到】
:
: all
: database
: 30TB in-memory db, huh? That's very impressive.
let's say I need to generate PDF file from some PDF template files.
then the PDF template files are the files my java code read frequently.
【在 m******t 的大作中提到】
:
: all
: database
: 30TB in-memory db, huh? That's very impressive.
m*t
24 楼
And so the templates will be cached by the OS. A linux kernel would
keep these files cached for as long as it has the memory to do so and
the access activities keep up.
【在 b******y 的大作中提到】
: ft... I gave example le.
: let's say I need to generate PDF file from some PDF template files.
: then the PDF template files are the files my java code read frequently.
m*t
25 楼
Dude, I don't know about your book, by mine, if something is
the result of a parsing, then it's no longer "static". 8-)
【在 g*****g 的大作中提到】
: Flexibility and portablity, below was a real app I worked in ex-job.
: Let's say you want to implement an web server
: that can display emails, all emails are in MIME format so
: you have to parse emails to get attachments in the first
: place. And of course you want to cache them to avoid parsing
: them again.
: Now use a pure java solution, you have the control of the eviction
: policy. You may give VIP members more cache space or longer
: expiry time for example. Your caching is portable, config the
: director
the result of a parsing, then it's no longer "static". 8-)
【在 g*****g 的大作中提到】
: Flexibility and portablity, below was a real app I worked in ex-job.
: Let's say you want to implement an web server
: that can display emails, all emails are in MIME format so
: you have to parse emails to get attachments in the first
: place. And of course you want to cache them to avoid parsing
: them again.
: Now use a pure java solution, you have the control of the eviction
: policy. You may give VIP members more cache space or longer
: expiry time for example. Your caching is portable, config the
: director
b*y
26 楼
a newbie question here: how to invoke OS
caching capacity from my java code? how to control OS caching,
say, LRU parameters, force to expire, etc, from java code?
【在 m******t 的大作中提到】
: Dude, I don't know about your book, by mine, if something is
: the result of a parsing, then it's no longer "static". 8-)
caching capacity from my java code? how to control OS caching,
say, LRU parameters, force to expire, etc, from java code?
【在 m******t 的大作中提到】
: Dude, I don't know about your book, by mine, if something is
: the result of a parsing, then it's no longer "static". 8-)
m*t
27 楼
(I realize I was answering your "how?" with "why?" the first
time around, only because I thought that was more relevant.)
Invoking it is simple - just use the darn files. :-)
Controlling it is obvious system-depedent. For instance on linux
you can (assuming your code can sudo) change a couple parameters
under /proc/sys/vm to control how much memory is used for file
caching, and when to flush etc.
It won't let you do per-file configuration obviously, so as I said
before, you don't have the same
【在 b******y 的大作中提到】
: a newbie question here: how to invoke OS
: caching capacity from my java code? how to control OS caching,
: say, LRU parameters, force to expire, etc, from java code?
g*g
28 楼
Depends on your requirement on portablitity.
I have this evil biased opinion operation is dumb.
And I don't like leaving a lot of stuff in their hands.
【在 m******t 的大作中提到】
:
: (I realize I was answering your "how?" with "why?" the first
: time around, only because I thought that was more relevant.)
: Invoking it is simple - just use the darn files. :-)
: Controlling it is obvious system-depedent. For instance on linux
: you can (assuming your code can sudo) change a couple parameters
: under /proc/sys/vm to control how much memory is used for file
: caching, and when to flush etc.
: It won't let you do per-file configuration obviously, so as I said
: before, you don't have the same
I have this evil biased opinion operation is dumb.
And I don't like leaving a lot of stuff in their hands.
【在 m******t 的大作中提到】
:
: (I realize I was answering your "how?" with "why?" the first
: time around, only because I thought that was more relevant.)
: Invoking it is simple - just use the darn files. :-)
: Controlling it is obvious system-depedent. For instance on linux
: you can (assuming your code can sudo) change a couple parameters
: under /proc/sys/vm to control how much memory is used for file
: caching, and when to flush etc.
: It won't let you do per-file configuration obviously, so as I said
: before, you don't have the same
b*y
29 楼
why not justified?
assume I have a PDF generator module that generates 1M
PDF from one of the 1000 PDF templates. It would be
very time consuming to load template from disk/DB each time.
(and assume there is no way to pre-process...)
【在 m******t 的大作中提到】
:
: (I realize I was answering your "how?" with "why?" the first
: time around, only because I thought that was more relevant.)
: Invoking it is simple - just use the darn files. :-)
: Controlling it is obvious system-depedent. For instance on linux
: you can (assuming your code can sudo) change a couple parameters
: under /proc/sys/vm to control how much memory is used for file
: caching, and when to flush etc.
: It won't let you do per-file configuration obviously, so as I said
: before, you don't have the same
assume I have a PDF generator module that generates 1M
PDF from one of the 1000 PDF templates. It would be
very time consuming to load template from disk/DB each time.
(and assume there is no way to pre-process...)
【在 m******t 的大作中提到】
:
: (I realize I was answering your "how?" with "why?" the first
: time around, only because I thought that was more relevant.)
: Invoking it is simple - just use the darn files. :-)
: Controlling it is obvious system-depedent. For instance on linux
: you can (assuming your code can sudo) change a couple parameters
: under /proc/sys/vm to control how much memory is used for file
: caching, and when to flush etc.
: It won't let you do per-file configuration obviously, so as I said
: before, you don't have the same
m*t
30 楼
Fair point - like I said "assuming you have sudo".
It's more of a political issue than a technical one
though.
【在 g*****g 的大作中提到】
: Depends on your requirement on portablitity.
: I have this evil biased opinion operation is dumb.
: And I don't like leaving a lot of stuff in their hands.
m*t
31 楼
I don't know about your exact situation so obviously
I can't just assert it. Now with that being said -
I don't see how the _reading_ of the template is
any different from reading any other file. IOW,
say your template is 1MB, unless the generator code
is incredibly idiotically written, all that it needs
to read from the disk is, well, 1MB. And it will
be cached by the OS, just like any other files.
Furthermore, if between two reads of the same template,
there are enough activities involving ot
【在 b******y 的大作中提到】
: why not justified?
: assume I have a PDF generator module that generates 1M
: PDF from one of the 1000 PDF templates. It would be
: very time consuming to load template from disk/DB each time.
: (and assume there is no way to pre-process...)
b*y
32 楼
ft... I forgot to assume the template files are not that big.
assume each template file is around 5k - 50k.
i'd say they are perfect candidate for an LRU cache of capacity 10.
【在 m******t 的大作中提到】
:
: I don't know about your exact situation so obviously
: I can't just assert it. Now with that being said -
: I don't see how the _reading_ of the template is
: any different from reading any other file. IOW,
: say your template is 1MB, unless the generator code
: is incredibly idiotically written, all that it needs
: to read from the disk is, well, 1MB. And it will
: be cached by the OS, just like any other files.
: Furthermore, if between two reads of the same template,
assume each template file is around 5k - 50k.
i'd say they are perfect candidate for an LRU cache of capacity 10.
【在 m******t 的大作中提到】
:
: I don't know about your exact situation so obviously
: I can't just assert it. Now with that being said -
: I don't see how the _reading_ of the template is
: any different from reading any other file. IOW,
: say your template is 1MB, unless the generator code
: is incredibly idiotically written, all that it needs
: to read from the disk is, well, 1MB. And it will
: be cached by the OS, just like any other files.
: Furthermore, if between two reads of the same template,
m*t
33 楼
If you anticipate no more than 10 templates being needed during
a short period of time, each of which no longer than 50k, trust me,
you don't have that much traffic for any choice of caching schemes
to even come close to mattering.
How many templates do you have in total? 1000? That's like 50MB.
Heck, just read all of them in one shot and be done with it. 8-)
【在 b******y 的大作中提到】
: ft... I forgot to assume the template files are not that big.
: assume each template file is around 5k - 50k.
: i'd say they are perfect candidate for an LRU cache of capacity 10.
Q*g
34 楼
when in JVM cache grows big, it drags the GC performance.
【在 g*****g 的大作中提到】
: Don't really have any experience with memcached.
: But I believe, ehCache and other Java Cache solution (JBoss Cache
: etc.) gives you more control and flexibility. e.g. You can configure
: ehCache as Hibernate's secondary Cache. And it's not intrusive.
: Also ehCache can run in your JVM. memcached is C based.
【在 g*****g 的大作中提到】
: Don't really have any experience with memcached.
: But I believe, ehCache and other Java Cache solution (JBoss Cache
: etc.) gives you more control and flexibility. e.g. You can configure
: ehCache as Hibernate's secondary Cache. And it's not intrusive.
: Also ehCache can run in your JVM. memcached is C based.
g*g
35 楼
You need to configure GC a little bit, allocate more memory
to tenured generation and make the GC algorithm less aggressive
should help.
【在 Q**g 的大作中提到】
: when in JVM cache grows big, it drags the GC performance.
to tenured generation and make the GC algorithm less aggressive
should help.
【在 Q**g 的大作中提到】
: when in JVM cache grows big, it drags the GC performance.
相关阅读
IntelliJ IDEA 7.01 is outjava新手弱问(eclipse)j2ee1.x是不是j2se1.x的superset?Simple question[合集] 谁能解释一下这里的protected specifierAnyone have experience with a Java Installer-generator ?求助：什么是.do file呀网页的Form中Post的问题 (转载)EJBGen祝java版的各位同修考虑今年年底换工作做J2EE的大侠们，能推荐一下比较好用的referrence website吗？[合集] Weiqi Gao: The Number One ill of Java[合集] Swing问题：Auto hide task bar 如何实现？JavaFX, anybody tried in real project?所有的project都不见了！爪哇的铜锈们新年好！[合集] 问一个很常见的过程的实现synchronization for countersJava虚拟磁盘问题