hubot-ibmcloud-objectstorage-crawler

A Node script that indexes Object Storage containers enabling a cognitive search. See the IBM Object Storage Service for more information.

Getting Started

• Usage
• Commands
• Cognitive Setup
• Development
• License
• Contribute

Usage

Steps for adding this to your existing hubot:

1. Run git clone https://github.com/ibm-cloud-solutions/hubot-ibmcloud-objectstorage-crawler
2. Run cd hubot-ibmcloud-objectstorage-crawler
3. Run npm install
4. Add the necessary environment variables for Object Storage, Watson Natural Language Classifier, Watson Visual Recognition, Watson Document Conversion, and Alchemy Language Services.

# Object Storage configuration
export HUBOT_OBJECT_STORAGE_AUTH_URL=<URL>
export HUBOT_OBJECT_STORAGE_USER_ID=<USER_ID>
export HUBOT_OBJECT_STORAGE_PASSWORD=<PASSWORD>
export HUBOT_OBJECT_STORAGE_PROJECT_ID=<PROJECT_ID>
export HUBOT_OBJECT_STORAGE_BLUEMIX_REGION=dallas

# Watson Natural Language Classifier configuration
export HUBOT_WATSON_NLC_URL=<NLC_URL>
export HUBOT_WATSON_NLC_USERNAME=<NLC_USERNAME>
export HUBOT_WATSON_NLC_PASSWORD=<NLC_PASSWORD>

# Watson Visual Recognition configuration
export HUBOT_VISUAL_RECOGNITION_API_KEY=<API_KEY>
export HUBOT_VISUAL_RECOGNITION_VERSION_DATE="2016-05-20"

# Watson Document Conversion configuration
export HUBOT_DOC_CONVERSION_USERNAME=<USERNAME>
export HUBOT_DOC_CONVERSION_PASSWORD=<PASSWORD>
export HUBOT_DOC_CONVERSION_VERSION_DATE=2015-12-15

# Alchemy Language configuration
export HUBOT_ALCHEMY_API_KEY=<API_KEY>

Development

Please refer to the CONTRIBUTING.md before starting any work. Steps for running this script for development purposes:

Configuration Setup to run Locally

1. Create config folder in root of this project.
2. Create env in the config folder
3. Add the environment variables described above to the env file.

Running Locally

• Run npm run start start the app that allows you to scan and index object storage
  • scan - scan objectstorage for changes since last training
  • index - index objectstorage using NLC
  • search - run search with the provided search string
  • exit - end this program
• Run npm run test to run automated tests
• Run npm run coverage-local to generate a code coverage report

License

See LICENSE.txt for license information.

Contribute

Please check out our Contribution Guidelines for detailed information on how you can lend a hand.