too many open files

[tcurdt]

I was trying to build your site but when serving the pages I get

$ make w
gostatic -w config
Rendering 0 changed pages of 292 total
Starting server at *:8000...
2016/07/04 22:37:34 http: Accept error: accept tcp [::]:8000: accept: too many open files; retrying in 5ms
2016/07/04 22:37:34 http: Accept error: accept tcp [::]:8000: accept: too many open files; retrying in 10ms

Building it and running python -m SimpleHTTPServer 8000 instead works fine though.

[piranha]

Running ulimit -S -n 4096 will fix that. gostatic opens all files and it takes time to GC them it seems, so it can blow with such an error.

[tcurdt]

OK, let's rephrase that then: why does it open all files and keep them open?

[piranha]

I think first part is obvious, but second one is not. :) I did look at why it opens them IIRC, but I didn't action on it. Not sure why it keeps them open though, that needs to be researched again it seems. :)

[tcurdt]

I think first part is obvious, but second one is not. :)

true :)

that needs to be researched again it seems.

care to re-open then?

[piranha]

Sure! :)) If you want to research/fix that, I would really appreciate it! If not, I guess I'll look at that sometime, but not sure when - a bit too busy lately with work and real-life stuff.

[nochso]

Had a look at the FAQ for fsnotify (which is imported by lib/watch.go)

How many files can be watched at once?

There are OS-specific limits as to how many watches can be created:

Linux: /proc/sys/fs/inotify/max_user_watches contains the limit, reaching this limit results in a "no space left on device" error.
BSD / OSX: sysctl variables "kern.maxfiles" and "kern.maxfilesperproc", reaching these limits results in a "too many open files" error.

As long as you're watching them, the files have to be open. It's easiest to increase the limit.

Alternatively you could use a "polling" approach which is way slower, but will work for any amount of files.

[tcurdt]

Well, that statement isn't true for all OS. On macOS it would be great to use FSEvents.

https://github.com/emcrisostomo/fswatch

But it sounds like there is a progress in supporting that with fsnotify.

golang/go#4068

[tcurdt]

...and specifically fsnotify/fsnotify#11