osx watcher limit

[b1lly]

Is there currently a work around for this osx watcher limit?

[nathany]

We currently use kqueue on OS X which is limited to 10,240 open file descriptors (number of files watched). I found this article to increase the limit but I haven't tried to do so myself:
http://krypted.com/mac-os-x/maximum-files-in-mac-os-x/

[b1lly]

hm... thanks for responding.

I'm currently creating one watcher that watches 28 unique directories. It's throwing the "to many files open" error on mac osx. This doesn't seem anywhere close to the limit? The directories aren't big at all, so I'm unsure why this would happen.

[nathany]

File descriptors are used for other things too, even if you're only using a few hundred. I'm not aware of any other limits besides # of files.

One thing you might check is to ensure you aren't watching hidden directories like .git. That could add significantly to the # of files.

fsnotify currently watches all the files in the directories you specify. I'm currently investigating what can be accomplished by just watching directories:
https://groups.google.com/d/msg/golang-dev/bShm2sqbrTY/vEvLBB3sIdkJ

It's also possible that someone will contribute an FSEvents adapter #54 which is geared for watching directory structures.

Sorry I don't have an immediate solution for you.

[nathany]

"The default per-process descriptor limit on current versions of OS X is extremely low (256!). "

From the Watchman README

[pifantastic]

Just to throw this out there, node's gaze (https://github.com/shama/gaze#errors) library detects EMFILE errors and fallsback to polling. Would it be terrible to do the same here? Maybe with an option to not fallback for people who can't allow the performance hit?

In my case, I want to know if any file in a directory changes, with the potential for large numbers of files. It looks like I'll need to build my own wrapper for fsnotify that falls back to polling. If I know that there is a potential to roll that functionality into fsnotify, I can build it with that in mind.

[nathany]

@pifantastic Based on the discussions around inclusion of os/fsnotify in standard library, I suspect a separate polling package would be best, but you can bring it up on the golang-dev list if you like.

The event structure and API are sure to change from what we have now: https://goo.gl/MrYxyA