Kristian's web log

January 26, 2010

Varnish best practices

Filed under: /dev/random — Tags: , , , , — kristian @ 16:21

A while ago I wrote about common Varnish issues, and I think it’s time for an updated version. This time, I’ve decided to include a few somewhat uncommon issues that, if set, can be difficult to spot or track down. A sort of pitfall-avoidance, if you will. I’ll add a little summary with parameters and such at the end.

1. Run Varnish on a 64 bit operating system

Varnish works on 32-bit, but was designed for 64bit. It’s all about virtual memory: Things like stack size suddenly matter on 32bit. If you must use Varnish on 32-bit, you’re somewhat on your own. However, try to fit it within 2GB. I wouldn’t recommend a cache larger than 1GB, and no more than a few hundred threads… (Why are you on 32bit again?)

2. Watch /var/log/syslog

Varnish is flexible, and has a relatively robust architecture. If a Varnish worker thread was to do something Bad and Varnish noticed, an assert would be triggered, Varnish would shut down and the management process would start it up again almost instantly. This is logged. If it wasn’t, there’s a decent chance you wouldn’t notice, since the downtime is often sub-second. However, your cache is emptied. We’ve had several customers contact us about performance-issues, only to realize they’re essentially restarting Varnish several times per minute.

This might make it sound like Varnish is unstable: It’s not. But there are bugs, and I happen to see a lot of them, since that’s my job.

An extra note: On Debian-based systems, /var/log/messages and /var/log/syslog is not the same. Varnish will log the restart in /var/log/messages but the actual assert error is only found in /var/log/syslog, so make sure you look there too.

The best way to deal with assert errors is to search our bug tracker for the relevant function-name.

3. Threads

The default values for threads is based on a philosophy I’ve since come to realize isn’t optimal. The idea was to minimize the memory footprint of Varnish. So by default, Varnish uses 5 threads per thread pool. By default, that’s 10 threads minimum. The maximum is far higher, but in reality, threads are fairly cheap. If you expect to handle 500 concurrent requests, tune Varnish for that.

A little clarification on the thread-parameters: thread_pool_min is the minimum number of threads for each thread pool. thread_pool_max is the maximum total number of threads. That means the values are not on the same scale. The thread_pools parameter can safely be ignored (tests have indicated that it doesn’t matter as much as we thought), but ideally having one thread_pool for each cpu core is the rule of thumb, if you want to modify it.

You also do not want more than 5000 as the thread_pool_max. It’s dangerous, though fixed in trunk. It’s also more often than not an indication that something else is wrong. If you find yourself using 5000 threads, the solution is to find out why it’s happening, not to increase the number of threads.

To reduce the startup time, you also want to reduce the thread_pool_add_delay parameter. ‘2′ is a good value (as opposed to 20 which makes for a slow start).

4. Tune based on necessity

I often look at sites where someone has tried to tune Varnish to get the most out of it, but taken it a bit too far. After working with Varnish I’ve realized that you do not really need to tune Varnish much: The defaults are tuned. The only real exception I’ve found to this is number of threads and possibly work spaces.

Varnish is – by default – tuned for high performance on the vast majority of real-life production sites. And it scales well, in most directions. By default. Do yourself a favor and don’t fix a problem which isn’t there. Of all the issues I’ve dealt with on Varnish, the vast majority have been related to finding out the real problem and either using Varnish to work around it, or fix it on the related system. Off the top of my head, I can really only remember one or two cases where Varnish itself has been the problem with regards to performance.

To be more specific:

  • Do not modify lru_interval. I often see the value “3600″. Which is a 180 000% (one hundred and eighty thousand percent) increase from the default. This is downright dangerous if you suddenly need the lru-list, and so far my tests haven’t been able to prove any noticeable performance improvement.
  • Setting sess_timeout to a higher value increase your filedescriptor consumption. There’s little to gain by doing it too. You risk running out of file descriptors. At least until we can get the fix into a released version.

So the rule of thumb is: Adjust your threads, then leave the rest until you see a reason to change it.

5. Pay attention to work spaces

To avoid locking, Varnish allocates a chump of memory to each thread, session and object. While keeping the object workspace small is a good thing to reduce the memory footprint (this has been improved vastly in trunk), sometimes the session workspace is a bit too small, specially when ESI is in use. The default sess_workspace is 16kB, but I know we have customers running with 5MB sess_workspace without trouble. We’re obviously looking to fix this, but so far it seems that having some extra sess_workspace isn’t that bad. The way to tell is by asserts (unfortunately), typically something related to “(p != NULL) Condition not true” (though there can obviously be other reasons for that). Look for it in our bug report, then try to increase the session workspace.

6. Keep your VCL simple

Most of your VCL-work should be focused around vcl_recv and vcl_fetch. That’s where you define the majority of your caching policies. If that’s where you do your work, you’re fairly safe.

If you want to add extra headers, do it in vcl_deliver. Adding a header in vcl_hit is not safe. You can use the “obj.hits” variable in vcl_deliver to determine if it was a cache hit or not.

You should also review the default vcl, and if you can, let Varnish fall through to it. When you define your VCL, Varnish appends the default VCL, but if you terminate a function, the default is never run. This is an important detail in vcl_recv, where requests with cookies or Authroization-headers are passed if present. That’s far safer than forcing a lookup. The default vcl_recv code also ensures that only GET and HEAD-requests go through the cache.

Focus on caching policy and remember that the default VCL is appended to your own VCL – and use it.

7. Choosing storage backend (malloc or file?)

If you can contain your cache in memory, use malloc. If you have 32GB of physical memory, using -smalloc,30G is a good choice. The size you specify is for the cache, and does not include session workspace and such, that’s why you don’t want to specify -smalloc,32G on a 32GB-system.

If you can not contain your cache in memory, first consider if you really need that big of a cache. Then consider buying more memory. Then sleep on it. Then, if you still think you need to use disk, use -sfile. On Linux, -sfile performs far better than -smalloc once you start hitting disk. We’re talking pie-chart-material. You should also make sure the filesystem is mounted with noatime, though it shouldn’t be necessary. On Linux, my cold-hit tests (a cold hit being a cache hit that has to be read from disk, as opposed to a hot hit which is read from memory) take about 6000 seconds to run on -smalloc, while it takes 4000 seconds on -sfile with the same hardware.  Consistently. However, your milage may vary with things such as kernel version, so test both anyway. My tests are easy enough: Run httperf through x-thousand urls in order. Then do it again in the same order.

Some of the most challenging setups we work with are disk-intensive setups, so try to avoid it. SSD is a relatively cheap way to buy yourself out of disk-issues though.

8. Use packages and supplied scripts

While it may seem easier to just write your own script and/or install from source, it rarely pays off in the long run. Varnish usually run on machines where downtime has to be planned, and you don’t want a surprise when you upgrade it. Nor do you want to risk missing that little bug we realized was a problem on your distro but not others. If you do insist on running home-brew, make sure you at least get the ulimit-commands from the startup scripts.

This is really something you want regardless of what sort of software you run, though.

9. Firewall and sysctl-tuning

Do not set “tw_reuse” to 1 (sysctl). It will work perfectly fine for everyone. Except thousands of people behind various NAT-based firewalls. And it’s a pain to track down. Unfortunately, this has been an advice in the past.

Avoid connection-tracking on the Varnish server too. If you need it, you’ll need to tune it for high performance, but the best approach is simply to not do connection-tracking on a server with potentially thousands of new connections per second.

10. Service agreements

(Service agreements are partly responsible for my salary, so with that “conflict of interest” in mind….)

You do not need a service agreement to run Varnish. It’s free software.

However, if you intend to run Varnish and your site is business critical, it’s sound financial advice to invest some money in it. We are the best at finding potential problems with your Varnish-setup before they occur, and solving them fast when they do occur.

We typically start out by doing a quick sanity-test of your configuration. This is something we can do fast, both with regards to parameters, VCL and system configuration. Some of our customers only contact us when there’s something horribly wrong, others more frequently to sanity-check their plans or check up on how to use varnisncsa for their particular logging tool and so on. It’s all up to you.

We also have a public bug tracker anyone can access and submit to. We do not have a private bug tracker, though there are bugs that never hit the public bug tracker – but that’s because we fix them immediately. Just like any other free software project, really. We have several public mailing lists, and we answer them to the best of our ability, but there is no guarantee and our time is far more limited. If you run into a bug, my work on other bugs will be postponed until your problems are solved. Better yet: if you run into something you don’t know is a bug, we can track it down.

A service agreement gives you saftey. And your needs will get priority when we decide where we want to take Varnish in the future.

We also offer training on Varnish, if you prefer not to rely on outside competence.

Oh, and I get to eat. Yum.

Summary

Keep it simple and clean. Do not use connection tracking or tw_reuse. Try to fit your cache into memory on a 64-bit system.

Watch your logs.

Parameters:

thread_pool_add_delay=2
thread_pools = <Number of cpu cores>
thread_pool_min = <800/number of cpu cores>
thread_pool_max = 4000
session_linger = 50
sess_workspace = <16k to 5m>

So if you have a dual quad core CPU, you would have 8 cpu cores. This would make sense: thread_pools=8, thread_pool_min=100, thread_pool_max=4000. The number 800 is semi random: it seems to cover most use-cases. I addedd session_linger into the mix because it’s a default in Varnish 2.0.5 and 2.0.6 but not in prior versions, and it makes good sense.

January 19, 2010

Real-time and hit-and-run based statistics for VSTS

Filed under: Varnish — Tags: , , , , — kristian @ 14:18

A while ago I wrote VSTS – the Varnish Stress Testing Suite. It’s not nearly as fancy as the name makes it sound: It’s a simple set of shell scripts that runs on a set of test servers and pounds Varnish under a couple of different scenarios. The idea is simple: Detect possible performance issues during the development cycle of Varnish by periodically testing the current code against the same well-established test.

It does the job, but could be far better. Specially when it comes to statistics.

The rrdtool-backed Python script is simply insufficient. As rrdtool likes to put data entry into a time-slot, and I don’t really operate in time slots, the data is both slow and imprecise. This is fine if you want data for every 5 minute throughout a year, but not if you want data collected anwhere from one time to ten times a day. Simply put: I want one data entry to represent one unit on the X-scale.

I also want to add “real-time” statistics. The current data is collected after each test, but I want data to be gathered throughout the relevant tests. This should be seperate from the other statistics. Hopefully, the system I end up with is flexible enough to allow me to plot all the datasets on the same graph, which should make any variations easy to spot.

Up until now I’ve only ever dealt with rrdtool when it comes to statistics (unless you count pre-rrdtool mrtg and the like), so I was hoping for some pointers before I start digging through what I suspect is a jungle of similar but different solutions. I’m comfortable working with most languages, and the primary concerns I’m looking at is implementation complexity and flexibility.

When I’m done with this batch of refactoring, I’ll do a proper writeup. However, what I’m most excited about is adding OpenSolaris into the target-platforms (should help clean out a few bugs that have been plaguing the Solaris-users for a while), adding better support for customized tests and improved robustness of VSTS.

January 13, 2010

Pushing Varnish even further

Filed under: Varnish — Tags: , , — kristian @ 21:33

A while ago I did a little writeup on high-end Varnish tuning, where I noted that I made our single core 2.2GHz Opteron reach 27k requests/second. This begged the questions as to how well Varnish scale with hardware. So I went ahead and tried to overload our quad-core Xeon at 2.4GHz. It would obviously take some extra fire power. At the very least, four times as much as the last batch of tests.

Hardware involved

Our main set of test servers for Varnish are called varnish1, varnish2, varnish3, varnish4, varnish6 and varnish7. These have mostly different software and hardware – which is done intentionally so we can perform tests under different circuimstances. We routinely run tests against Varnish2 and Varnish4, which run CentOS and FreeBSD, respectively. For my last test, I used Varnish2 as the server and the remaining servers as test nodes. By any normal math, I would need  about 4 times more fire power to overload a 2.4GHz Quad core, compared to a single core Opteron at 2.2GHz.

To sum it up as far as this round of tests go:

  • Varnish1 – Single core Opteron
  • Varnish2 – Single core Opteron at 2.2GHz (used in the last round of tests)
  • Varnish3 – Single core Xeon (if I’m not much mistaken). It’s also the nginx server used as backend, but that just means 1 request every X minutes.
  • Varnish4 – Single core Opteron (FreeBSD)
  • Varnish6 – Dual core Xeon of some kind
  • Varnish7 – Quad-core Xeon at 2.4GHz

So I needed more power. As it happens, we do alot of training and we have three classrooms full of computers for students, and I borrowed two of these class rooms, adding the following to the mix:

  • 10 x single core Pentium Celerons at 2.9x GHz
  • 10 x Core 2 Duos at 2.4ish GHz

As you might notice – a large part of the challenge when you want to test Varnish is getting your test systems to keep up.

Basic test procedures

Same as last time, more or less: 1 byte pages and httperf. I’ve tried ab, siege and curl… And they simply do not offer the raw power of httperf combined with the control – if anyone cares to enlighten me on how to get the most out of them, then I’m more than willing to listen.

Ideally I wanted to test with 10 requests for each connection, and with mixed data set size. As it turns out, I ended up using 100 requests / second and bursting all of the requests, which is far from realistic. More on this later.

I have an intricate script system for the nightly tests, but that’s a story for an other time. For these tests I simply used clusterssh to replicate my input on 37ish shells. This has allowed me to instantly test identical setups on all the nodes, and to quickly review what their status is. I probably ran a thousand or more different variants of the same test this time around.

I’ve used varnishstat to monitor the request rate and other relevant stats, and top to monitor general load.

The backend I use is hosted on varnish3, which runs nginx and a simple rewrite to ‘current.txt’, which for this occasion was linked to a 1byte file.

Results

Varnish uses alot of threads, and as such, when it does finally saturate the CPU, the load average will skyrocket. On the last test, Varnish2 had a load of 600-700. During this load, Varnish2 would use 10-15 seconds to start ‘top’.

During this round of tests I had roughly 87GHz worth of clients, spread over 25 physical computers. All of the tests systems were running at full load. Varnish7 had a load average around 45. Logging in and starting top was close to instant. And Varnish was serving 143k requests per second.

Based on the load and general snappiness, I think it is safe to conclude that while Varnish was close to the breaking point, it hadn’t actually reached it. To put it simply: My clients were not fast enough. Before I told httperf to burst 100 requests for each connection, Varnish was serving 110-120k requests per second with a load less than 1.0, and the clients were still using all their fire power. I ended up stress testing my clients. Dammit.

However, as I came fairly close to the breaking point, I still believe there are a few interesting things to look at.

The scaling nature of Varnish

It’s very rare that you can see an application scale so well just by throwing cpu power and cpu cores at it. Varnish essentially didn’t get affected at all by the extra work needed to synchronize work on 4 cpu cores. In fact, if you look at the math, the raw performance on 4 cpu cores was actually BETTER than on one cpu core, when you look at it on a cycle-by-cycle.

I think it’s reasonably safe to say that when it comes to raw performance, we’ve nailed it with Varnish.

In fact, scaling Varnish is far more difficult when you increase your active data set beyond physical memory. Or when you introduce latency. Or when when you have a low cache hit rate. Or any other number of corner cases. There will always be bottlenecks.

What you can learn from this is actually simple: Do not focus on the CPU when you want to scale your Varnish setup. I know it’s tempting to buy the biggest baddest server around for a high-traffic site, but if your active data set can fit within physical memory and you have a 64-bit CPU, Varnish will thrive. And for the record: All CPU-usage graphs I’ve seen from Varnish installations confirm this. Most of the time, those sexy CPUs are just sitting idle regardless of traffic.

Myths and further research

Since I didn’t reach the breaking point, there’s not much I can say conclusively. However, I can repeat a few points.

Adjusting the lru_interval had little impact regardless of data set and access patterns. If I repeat this often enough, perhaps I’ll stop seeing new installations with an lru_interval of 3600: DO NOT SET lru_interval TO 3600. There. I didn’t even add the usual “unless you know what you are doing” part. I might’ve explained it before, but the problem is that it leaves you with a really badly sorted lru-listed that will cause Bad Things once you need to lru-nuke something. Possibly really really bad things. Like throwing out the 200 most popular objects on your site at the same time.

And the size of your VCL has little impact on the performance. I have not tested this extensively, but I’ve never registered a difference, and since your cpu will be idle most of the time anyway, you should NOT worry about CPU cycles in VCL.

An otherimportant detail is that your shmlog shouldn’t trigger disk activity. On my setup, it didn’t sync to disk to begin with, but you may want to stick it on a tmpfs just to be sure. I suspect this has improved throughout the 2.0-series of Varnish, but it’s an easy insurance. Typically the shmlog is found in /usr/var/varnish, /usr/local/var/varnish or similar (”ls /proc/*/fd | grep _.vsl” is the lazy way to find it).

I tried several different settings of thread pools, acceptors, listen depth, shmlog parameters, rush exponent and such, but none of it revealed much – most likely because I never pressured Varnish enough. This will be what I want to investigate further. But it should tell you something about how far you have to go before these obscure settings start to matter.

Feedback wanted

I figure this must be some sort of record, but I’m interested in what sort of numbers others have seen or are seeing. Have anyone even come close to the numbers above – synthetic or otherwise – from a single server? Regardless of software or hardware? This is not meant as a challenge or boast, but I’m genuinely curious on what sort of traffic people are able to push. I’m interested in more “normal” requests rates too – I’m a sucker for numbers. What are you seeing on your site? Have you had scaling issues?

January 8, 2010

Hitpass objects and Varnish

Filed under: Varnish — Tags: , — kristian @ 10:39

Yesterday presented an interesting case of “what the hey is going on!?” for a customer of ours. The problem presented itself as a front page that sporadically needed 40 to 50 seconds to purge. After a brief hunt I discovered a somewhat interesting set of coincidences.

Before I go into detail, I need to tell you what a hitpass object is and why it’s needed.

Varnish can pass objects – that is to say, deliver an object without storing it in the cache – at two different stages in the request handling. The first and simplest stage you can pass at is before you have contacted the web server. This can be because you know that every page under a specific url should not be cached, or that a cookie is present and needed. This is done in vcl_recv. The slightly more complex stage to  purge at is after Varnish has gotten a reply from the backend, typically because it sent a “Set-Cookie” header or otherwise had headers that indicate that the page isn’t cacheable. This is done in vcl_fetch.

Normally Varnish tries to only request a specific object once, since it will look the same to all users there is no reason to make the web server generate it multiple times, and it’s definitely not faster either. When you are passing an object, every client request has to go to the web server, so serializing the requests for the same url is not wanted.

If you pass in recv, it’s easy for Varnish to avoid this serialization . It already knows that the next request for the same URL will have to go to the backend. Varnish makes an “anonymous” object that is never entered into the cache. A very simple and effective design.

But in vcl_fetch, things are not so easy. Varnish can’t predict a pass in vcl_fetch ahead of time, so it has to serialize these requests as if it was being cached. This is obviously not a desirable situation. Enter the hitpass object.

To avoid serialized requests after a pass in vcl_fetch, Varnish makes a hitpass object which is entered into the cache for ‘obj.ttl’ period of time, which simply says that this url will be passed for the next obj.ttl period of time. This looks almost just like a normal object in the cache, but it has no content, just a flag telling Varnish to fetch it from a backend. And thus Varnish can perform parallel requests again, because it knows the request will not be cached before it contacts the web server. The moment Varnish sees a hitpass object, it will dereference (and thus unlock) the hitpass object and make an anonymous one, entering the same code-path as if it was a pass in vcl_recv.

Now, back to my original story and purging. What’s it got to do with hitpass objects? Well, the purges on this particular site is done in vcl_hit/vcl_miss by resetting ttl to 0, effectively expiring the object. The problem was that at some point, a header on the relevant front page had told Varnish not to cache. So a hitpass object was made, and it had a ttl of roughly 4 days. So the VCL never hit vcl_hit or vcl_miss – where the purging was being done – but instead went to vcl_pass and directly to the backend, which kindly generated the page even though it just got a PURGE request, which is a bit strange, but not all that strange. What’s worse, of course, is that the front page wasn’t cached at all for this period of time. As it turns out, Varnish also has a minor issue which I also discovered yesterday, in that it doesn’t actually purge hitpass objects correctly, at least not in this particular version of Varnish.

And if you are thinking “why not set the ttl to 0 in vcl_pass if you see a PURGE?”, even if we allowed you to modify obj.ttl in vcl_pass, it wouldn’t be the ttl of the hitpass object, but the anonymous object. This is also why you can’t simply decide to start caching it again – when you reach vcl_fetch after hitting a hitpass object, the object you are working on is anonymous, and never entered into the cache.

My solution? Add an ‘x’ to the cache hash for the particular url/host combination for an immediate “fix”. It was either that or restarting Varnish at peak hour. Oh, and add some sanity to the VCL to reduce the effect of this in the future.

Recommendations

Understand hitpass objects. If you are doing ‘pass’ in vcl_fetch, remember that the TTL matters. You may want to switch ‘pass’ in vcl_fetch with your own pass subroutine which just sets the ttl to 1 minute for instance, and use that function consistently. That way, you avoid long-lived hitpass objects, but still get the benefit of them when you want them.

Also: If you are using the artificial http “PURGE”-method to set ttl to zero in vcl_hit, don’t forget to check for it in vcl_pass. You probably want to throw a 503-error in vcl_pass if you see a PURGE request.

January 4, 2010

The problem with piracy

Filed under: /dev/random — Tags: , , , — kristian @ 00:27

If you are paying attention to the news, you will be aware of the rising piracy-problem. International authoroties have tried desperately to stop the piracy, but so far little has helped. It has had large economic impacts, both in stolen goods and in the cost of the war on piracy.

As it turns out, the core of the prolem lies in the nature of the piracy: Piracy is taking place because the people feel that there is little to loose. The environment in which the piracy takes place is one where the pirates and the community around them is constantly bullied and given little hope of improvement. The situation has only gotten worse over the years, and the efforts to stop the piracy is ultimately focused on enforcing the existing laws instead of solving the problems of the society. And while authoroties might be able to quench the piracy, it comes at a cost too: The society is again left to improve itself without any help real from the outside because the rest of the world is happy with the situation as it is and doesn’t want to stir things up.

For those who haven’t been paying attention to the news, I’m talking about Somalia, a country that was ranked 161st on UNs list of roughly 180 countries based on it’s “human development index”. That is, the last time it was included in the ranking, which was in 2001.

So called illegal file sharing and the war on sharing has many similarities with the story above. Beyond the obvious, there’s insistance on calling both pirates and file sharers for immoral outcasts of society that only prey on their surroundings and doesn’t give anything back. If it could be linked to terrorists; even better. And yes, according to the propaganda-movie at the end of my “House MD” dvd, file sharing funds terrorism. I’m not entirely sure how, but hey, I bet it’s a good argument. The reality is – of course – different. It also states that it costs jobs…. well… isn’t that a GOOD thing for me? I don’t want to pay a bunch of guys for sitting around doing nothing. The distribution networks of today is terribly inefficient. The concept of being “out of stock” should’ve gone away with the invention of the DVD. And the idea that it costs jobs is also strange, because all the propaganda-makers and lawyers wouldn’t have jobs if it wasn’t for the war on sharing.

There is one big difference: Piracy is carried out by means of violence. Pirates steal actual goods. File sharing doesn’t actually have any material costs, nor is it by any conclusive and through research possible to say that it even reduces sales. In fact, I’m sure I’m not the only one who suspect that file sharing increases sales. I own over 350 dvds and blueray movies now. I bought several of them after I had seen them. Why? Two reasons: 1. Because it’s right. 2. Because I liked the movie.

An other difference is that the somali pirates struggle to improve the conditions of a geographicly limited society, whereas file sharers operate in a global society. All in all it’s not that different, though: File sherers don’t get any real out-side help either, and are always on the defensive, regardless of how they “fight”. It is hard for outsiders to understand the society that the pirates live in, just as it’s hard for someone who wasn’t brought up in the digital age to understand how wrong the war against file sharing feels.

I do not condone piracy. But I can understand why it’s taking place and I belive it’s extermely cynical to use brute force to stop it without any hint of a public debate as to the soiciological conditions that allowed it in the first place. In the same way that I don’t like the idea of breaching copyrights through illegal file sharing, but I think it’s extermely cynical – and naive – to think that the real problem can be stopped by means of authority, laws and technology created to maintain the status quo in an ever-changing global class struggle.

It’s also very hard to talk openly about illegal file sharing: Admitting the act is difficult because it’s illegal. So who’s to speak for the file sharers? What we need, is asylum for file sharers to enable a proper public debate….. But that’ll never happen.

Oh, and let’s quit calling it “piracy” when we mean one of or all of:

  • Copying a movie to your friend
  • Sharing a movie to strangers on the internet
  • Buying an obviously illegaly copied movie on the streets of some foreign country (this you should never do – ever. Get it on the net instead, since this really is for-profit crime)
  • Downloading a movie or series off the net because it’s not available where you live.

In the end, this is what I want:

  • Movies and music
  • The ability to pay for said movies and music
  • Market models designed around current technology
  • NOT technology designed around and designed to enforce the current market model (ie: DRM)
  • Freedom
  • Fairness

If you can sell 100 copies of the same movies for one tenth of the price you could sell ten movies for (distribution costs included) then why not do it? That’s what copyright is for: Enabling the public to get access to professional copyrighted works. It is not to protect the thoughts of the copyright holder. You have a skull to protect your intellectual property, and we have laws against bashing your skull in. And if you need technology to protect your intellectual property, then wear a god damned helmet. In the mean time, I don’t use hemlets to protect from robbers, so don’t force a helmet onto my blyeray disc movies.

I could go on and on with this rhetoric and endless digressions to underline the point… But what’s the point…. Money talks, after all, but not my money, it seems.

Powered by WordPress