-
Notifications
You must be signed in to change notification settings - Fork 0
License
jdost321/hdfs-xrootd-fallback-proj
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
hdfs-xrootd-fallback-proj ========================= The HDFS XRootD Fallback system enables relaxed local Hadoop replication by utilizing global redundancy provided by the XRootD Federation. This system provides exception handling at the block level, a cache to locally store repaired blocks retrieved from the Federation, and the ability to re-inject the repaired blocks back into Hadoop. The system currently is composed of 3 packages: hdfs-xrootd-fallback hdfs-xrootd-fbmon hdfs-xrootd-healer Installation instructions can be found here: http://www.gled.org/cgi-bin/twiki/view/Main/HdfsXrootd hdfs-xrootd-fallback ==================== The HDFS XRootD Fallback package is installed on every datanode in the Hadoop cluster and accesses blocks on demand via XRootD Cache on failed read exceptions hdfs-xrootd-fbmon ================= The HDFS XRootD Fallback Monitor is a UDP listener that logs incomming messages sent from the datanodes whenever a fallback is triggered. hdfs-xrootd-healer ================== The HDFS XRootD Healer is installed on the XRootD Cache node and periodically compares corrupt blocks in Hadoop with blocks stored in the cache. It re-injects the repaired blocks once they are fully cached.
About
No description, website, or topics provided.
Resources
License
Stars
Watchers
Forks
Packages 0
No packages published