-
Notifications
You must be signed in to change notification settings - Fork 0
/
hdfs-xrootd-fallback-proj.spec
196 lines (161 loc) · 6.54 KB
/
hdfs-xrootd-fallback-proj.spec
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
%define lib_hadoop_dirname /usr/lib
%define lib_hadoop %{lib_hadoop_dirname}/hadoop
%define lib_hdfs %{lib_hadoop_dirname}/hadoop-hdfs
Name: hdfs-xrootd-fallback-proj
Version: 1.0.2
Release: 1%{?dist}
Summary: Tools to enable relaxed local Hadoop replication
Group: System Environment/Daemons
License: BSD
URL: http://www.gled.org/cgi-bin/twiki/view/Main/HdfsXrootd
Source0: %{name}-%{version}.tar.gz
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
BuildRequires: java7-devel
BuildRequires: pcre-devel
BuildRequires: xrootd-client-devel >= 4.0.4
BuildRequires: hadoop-hdfs >= 2.0.0+545-1.cdh4.1.1.p0.19.osg
%description
The HDFS XRootD Fallback system enables relaxed local Hadoop replication by
utilizing global redundancy provided by the XRootD Federation. This system
provides exception handling at the block level, a cache to locally store
repaired blocks retrieved from the Federation, and the ability to re-inject the
repaired blocks back into Hadoop.
%package -n hdfs-xrootd-fallback
Summary: Hadoop extension to interface with XRootD for block-level read error prevention
Group: System Environment/Daemons
Requires: pcre
Requires: xrootd-client-libs >= 4.0.4
Requires: hadoop-hdfs >= 2.0.0+545-1.cdh4.1.1.p0.19.osg
Requires: hadoop-hdfs-fuse >= 2.0.0+545-1.cdh4.1.1.p0.19.osg
%description -n hdfs-xrootd-fallback
The HDFS XRootD Fallback package is installed on every datanode in the Hadoop
cluster and accesses blocks on demand via XRootD Cache on failed read exceptions
%package -n hdfs-xrootd-healer
Summary: Daemon that periodically re-injects cached blocks back into hadoop
Group: System Environment/Daemons
%if 0%{!?el5:1}
BuildArch: noarch
%endif
Requires: hadoop-hdfs-fuse >= 2.0.0+545-1.cdh4.1.1.p0.19.osg
Requires: python
%description -n hdfs-xrootd-healer
The HDFS XRootD Healer is installed on the XRootD Cache node and periodically
compares corrupt blocks in Hadoop with blocks stored in the cache. It re-injects
the repaired blocks once they are fully cached.
%package -n hdfs-xrootd-fbmon
Summary: Fallback monitoring daemon
Group: System Environment/Daemons
%if 0%{!?el5:1}
BuildArch: noarch
%endif
Requires: perl
Requires: perl-Proc-Daemon
%description -n hdfs-xrootd-fbmon
The HDFS XRootD Fallback Monitor is a UDP listener that logs incomming messages
sent from the datanodes whenever a fallback is triggered.
%prep
%setup -q
%build
export JAVA_HOME=%{java_home}
%configure \
HADOOP_HOME=%{lib_hadoop} \
HADOOP_HDFS_HOME=%{lib_hdfs} \
CPPFLAGS=-I/usr/include/xrootd
make %{?_smp_mflags}
%install
rm -rf %{buildroot}
make install DESTDIR=%{buildroot}
# rhel specific dirs
mkdir -p %{buildroot}/%{_sysconfdir}/sysconfig
mkdir -p %{buildroot}/%{_initrddir}
mkdir -p %{buildroot}/%{_sysconfdir}/cron.d
mkdir -p %{buildroot}/%{_sysconfdir}/logrotate.d
# rhel specific healer stuff
mkdir -p %{buildroot}/%{_sysconfdir}/hdfs-xrootd-healer
install -p -m 644 %{buildroot}/%{_datadir}/hdfs-xrootd-healer/hdfs-xrootd-healer.cfg \
%{buildroot}/%{_sysconfdir}/hdfs-xrootd-healer
install -p %{buildroot}/%{_datadir}/hdfs-xrootd-healer/hdfs-xrootd-healer.init \
%{buildroot}/%{_initrddir}/hdfs-xrootd-healer
install -p -m 644 %{buildroot}/%{_datadir}/hdfs-xrootd-healer/hdfs-xrootd-healer.cron \
%{buildroot}/%{_sysconfdir}/cron.d/hdfs-xrootd-healer
install -p -m 644 %{buildroot}/%{_datadir}/hdfs-xrootd-healer/hdfs-xrootd-healer.logrotate \
%{buildroot}/%{_sysconfdir}/logrotate.d/hdfs-xrootd-healer
mkdir -p %{buildroot}/%{_localstatedir}/lock/hdfs-xrootd-healer
#rhel specific fbmon stuff
install -p -m 644 %{buildroot}/%{_datadir}/hdfs-xrootd-fbmon/hdfs-xrootd-fbmon.sysconfig \
%{buildroot}/%{_sysconfdir}/sysconfig/hdfs-xrootd-fbmon
install -p %{buildroot}/%{_datadir}/hdfs-xrootd-fbmon/hdfs-xrootd-fbmon.init \
%{buildroot}/%{_initrddir}/hdfs-xrootd-fbmon
install -p -m 644 %{buildroot}/%{_datadir}/hdfs-xrootd-fbmon/hdfs-xrootd-fbmon.logrotate \
%{buildroot}/%{_sysconfdir}/logrotate.d/hdfs-xrootd-fbmon
%clean
rm -rf %{buildroot}
%post -n hdfs-xrootd-fallback -p /sbin/ldconfig
%postun -n hdfs-xrootd-fallback -p /sbin/ldconfig
%post -n hdfs-xrootd-healer
if [ $1 = 1 ];then
/sbin/chkconfig --add hdfs-xrootd-healer
fi
%preun -n hdfs-xrootd-healer
if [ $1 = 0 ];then
/sbin/service hdfs-xrootd-healer stop >/dev/null 2>&1 || :
/sbin/chkconfig --del hdfs-xrootd-healer
fi
%pre -n hdfs-xrootd-fbmon
getent group hdfsfbmon >/dev/null || groupadd -r hdfsfbmon
getent passwd hdfsfbmon >/dev/null || \
useradd -r -g hdfsfbmon -d %{_datadir}/hdfs-xrootd-fbmon -s /sbin/noligin \
-c "HDFS XRootD Fallback Monitor User" hdfsfbmon
exit 0
%post -n hdfs-xrootd-fbmon
if [ $1 = 1 ];then
/sbin/chkconfig --add hdfs-xrootd-fbmon
fi
%preun -n hdfs-xrootd-fbmon
if [ $1 = 0 ];then
/sbin/service hdfs-xrootd-fbmon stop >/dev/null 2>&1 || :
/sbin/chkconfig --del hdfs-xrootd-fbmon
fi
%files -n hdfs-xrootd-fallback
%defattr(-,root,root,-)
%doc README LICENSE NEWS
%{lib_hadoop}/client/hdfs-xrootd-fallback-%{version}.jar
%{_libdir}/libXrdBlockFetcher.so*
%{_sysconfdir}/hadoop/conf.osg/xfbfs-site.xml
%files -n hdfs-xrootd-healer
%defattr(-,root,root,-)
%doc README LICENSE NEWS
%{_sbindir}/hdfs-xrootd-healer
%{_datadir}/hdfs-xrootd-healer
%dir %{_sysconfdir}/hdfs-xrootd-healer
%config(noreplace) %{_sysconfdir}/hdfs-xrootd-healer/hdfs-xrootd-healer.cfg
%{_initrddir}/hdfs-xrootd-healer
%{_sysconfdir}/cron.d/hdfs-xrootd-healer
%config(noreplace) %{_sysconfdir}/logrotate.d/hdfs-xrootd-healer
%dir %{_localstatedir}/lock/hdfs-xrootd-healer
%dir %{_localstatedir}/log/hdfs-xrootd-healer
%files -n hdfs-xrootd-fbmon
%defattr(-,root,root,-)
%doc README LICENSE NEWS
%{_libexecdir}/hdfs-xrootd-fbmon
%{_datadir}/hdfs-xrootd-fbmon
%config(noreplace) %{_sysconfdir}/sysconfig/hdfs-xrootd-fbmon
%{_initrddir}/hdfs-xrootd-fbmon
%config(noreplace) %{_sysconfdir}/logrotate.d/hdfs-xrootd-fbmon
%attr(-,hdfsfbmon,hdfsfbmon) %dir %{_localstatedir}/log/hdfs-xrootd-fbmon
%attr(-,hdfsfbmon,hdfsfbmon) %dir %{_localstatedir}/run/hdfs-xrootd-fbmon
%changelog
* Thu Nov 4 2015 Jeff Dost <[email protected]> - 1.0.2-1
- Release v1.0.2
* Thu Mar 9 2015 Jeff Dost <[email protected]> - 1.0.1-1
- Release v1.0.1
* Thu Mar 2 2015 Jeff Dost <[email protected]> - 1.0.0-5
- Add fbmon rpm
* Thu Dec 24 2014 Jeff Dost <[email protected]> - 1.0.0-4
- Add healer rpm
* Thu Dec 4 2014 Jeff Dost <[email protected]> - 1.0.0-3
- Rebuild against xrootd4
* Thu Apr 3 2014 Jeff Dost <[email protected]> - 1.0.0-2
- Bug fix in libXrdBlockFetcher.so
* Fri Feb 14 2014 Jeff Dost <[email protected]> - 1.0.0-1
- Initial release