Tracking: MVP checklist

[hopeyen]

This issue should track all the items needed for a minimal viable product.

More updated details can be found in Feature_checklist.md.

General

• Standarlise subfile/chunkfile manifest formats Standarlise subfile.yaml manifest formats #2
• Draft out GRC Docs: Draft GRC #25
• Better naming Refactor: Adjust to a better fitting namespace #18

User experience

• Dataset Discovery on a marketplace
  • File Discovery and matching across datasets Feat: File Discovery and matching across datasets #19
    • Search Functionality
  • Dataset Listing from server
• Matching Algorithm Feat: Client wallet connection and dumb TAP payment #21
  • User Preference Analysis (price per byte, response rate)
  • Transaction History Utilization

Subfile transfer

• work with Horizon for data service interfaces
• Server
  • port into indexer-service framework (should take care of TAP receipt handling)
  • add cost model scheme, allow updates for pricing per byte Feat: Server cost model #20
• Use generic path to be compatible with cloud storages Spike: Generalize storage paths #15

Subfile Client

• Verifiable Payment Feat: Client wallet connection and dumb TAP payment #21
  • Take private key/mneomic for wallet connections
  • take budget for the overall subfile
    • construct receipts using budget and chunk sizes
    • add receipt to request
• Parallelize requests Perf: Client parallelize requests for one file at a time #16
• Multiple connections (HTTPS over HTTP2)
• Continually requesting missing pieces until the complete file is obtained Feat: Downloader send requests based on missing chunks #22

Testing and Deployment

• Service custom errors Refactor: Custom service error #23
• Track metrics Feat: Metrics for the server #24
• Track code test coverage
• Deployment Planning
  • Add deployment options (docker + binary)
  • Internal Infrastructure Support
  • Reach out to key users for early feedbacks
• Documentation and Support
  • Support System Setup (Channel)
  • Tutorials, FAQs

Beyong MVP scope

• Multiple hashing options/scheme; Taking file sizes and number of files, analyze performance and memory of using merkle tree vs hash list