File TODO changed (mode: 100644) (index 7f4898e..19a4f8c) |
1 |
1 |
== Where I stopped last time == |
== Where I stopped last time == |
|
2 |
|
[ ] unit testing for forgot_link - seems is not working! |
|
3 |
|
[ ] Visit all functions in git.int to not pass unescaped references to bash. |
|
4 |
|
[ ] The verification of user-provided CSRF crumbs with the expected value did |
|
5 |
|
not use a constant-time comparison algorithm, potentially allowing |
|
6 |
|
attackers to use statistical methods to determine valid CSRF crumbs |
|
7 |
|
using brute-force methods. |
|
8 |
|
[ ] clone seems to be done by git and not by ssh! |
|
9 |
|
[ ] build: environment lists must be with checkboxes |
|
10 |
|
[ ] build: when a client connects, add the environments to the env table. |
|
11 |
|
Allow to expire? or allow the admin to do it? |
|
12 |
|
[ ] log2listing: seems I do not add labels per commit but only at the end! |
|
13 |
|
[ ] Oracle: libsepol.print_missing_requirements: rocketgit's global |
|
14 |
|
requirements were |
|
15 |
|
not met: type/attribute passwd_file_t (No such file or directory). |
|
16 |
|
libsemanage.semanage_link_sandbox: Link packages failed (No such file |
|
17 |
|
or directory). /usr/sbin/semodule: Failed! |
|
18 |
|
[ ] Document how to start builder.sh (from cron or let it read the conf file) |
2 |
19 |
[ ] |
[ ] |
3 |
20 |
|
|
|
21 |
|
++ |
|
22 |
|
Allow workes to connect, but admin must validate them before being able |
|
23 |
|
to build something. At the same time, also set the auth key? |
|
24 |
|
Also, client must generate a ssh key and the public part must be sent to |
|
25 |
|
the server. After validation, the user can clone by ssh! |
|
26 |
|
I think is better to create an archive of the repo and transfer it by |
|
27 |
|
the same protocol. Not clear if I should use "git clone" or "git archive". |
|
28 |
|
++ |
|
29 |
|
|
4 |
30 |
== BEFORE NEXT RELEASE == |
== BEFORE NEXT RELEASE == |
|
31 |
|
[ ] sql: use somehow the rg_sql_conn[]['app'] |
|
32 |
|
Keep in mind that some postgresql version (8) does not support |
|
33 |
|
application_name= connection parameter. |
|
34 |
|
But, we may use: set application_name = 'newappname'; |
|
35 |
|
[ ] low: build: auto install dependencies based on .spec/debian files. |
|
36 |
|
[ ] Show labels also in the commits list? |
|
37 |
|
[ ] wh: Add "Change labels" action for a hook. |
|
38 |
|
[ ] build: Run a build client on Amazon? |
|
39 |
|
[ ] wh: last_output: add repo/branch/etc.! |
|
40 |
|
[ ] build: allow admin to set the environments? |
|
41 |
|
[ ] wh: add labels to repo that will be passed to web hooks, and reflected in |
|
42 |
|
##...## tags. |
|
43 |
|
[ ] sec: is it safe to store serialize stuff in db? |
|
44 |
|
[ ] curl: remove return headers because we also trace the execution and |
|
45 |
|
doubles the last output. |
|
46 |
|
[ ] wh: have_events -> list of supported events |
|
47 |
|
[ ] unit test: lambda - not clear for what events |
|
48 |
|
[ ] build: user must be able to add custom hooks to builds |
|
49 |
|
[ ] build: add some tags for label_ok/nok: #date#, #time#, #elapse# |
|
50 |
|
##worker_name## etc. |
|
51 |
|
[ ] build: add build time to be sent to builder.php |
|
52 |
|
[ ] build hook: provide a way to export an archive (provide a list of files) |
|
53 |
|
[ ] build hook: inside virtual machine, switch to a non-privileged user. |
|
54 |
|
[ ] ci: check codeship.io, circleci |
|
55 |
|
[ ] Not clear how to make an event to depend on a preceding one. |
|
56 |
|
Also, how to execute next hook when one finished? |
|
57 |
|
Should I do ci in parallel? |
|
58 |
|
[ ] Add a 'build' hook: |
|
59 |
|
Support docker/lxc/kvm. |
|
60 |
|
We must fork to not block events processing. But we should not |
|
61 |
|
spawn too many builders. I do not have support for postponing an event. |
|
62 |
|
Probably we should add the job in a queue. The queue is run by root. |
|
63 |
|
Result will be put in a status file. |
|
64 |
|
How should we prepare the job? We need the URL to clone (optimize this) |
|
65 |
|
and a command to be run inside. Also we need an image to be used to |
|
66 |
|
do ths build. |
|
67 |
|
--net=none/bridge |
|
68 |
|
--user - check |
|
69 |
|
--ulimit |
|
70 |
|
chcon -Rt svirt_sandbox_file_t /path/that/will/exported/as/volume |
|
71 |
|
|
|
72 |
|
I must find a way to allow the cloning for private repos (a key?). |
|
73 |
|
Or a side channel to transfer the files. |
|
74 |
|
What command should I use to do the cloning? |
|
75 |
|
I think the best way is to clone - and to allow the cloning |
|
76 |
|
done by builders. Mark some users as builders and allow access |
|
77 |
|
automatically? How should I give the rights? Maybe the function that |
|
78 |
|
loads the rights to automatically add Fetch rights if user is marked |
|
79 |
|
as 'builder'? |
|
80 |
|
|
|
81 |
|
We need to specify the architecture. |
|
82 |
|
|
|
83 |
|
Sometime we may use kvm - but we must prepare a nice build machine |
|
84 |
|
with a lot of build tools - no problem! Ask for dependencies? |
|
85 |
|
Look for a .spec file (for Fedora)? |
|
86 |
|
I must switch to a user to do the build. Maybe using a sandbox? |
|
87 |
|
How can we list the images available? |
|
88 |
|
We may define them in admin section! |
|
89 |
|
Including base image. |
|
90 |
|
Have different queue for different types of build scripts (docker, libvirt) |
|
91 |
|
In 'admin' section, add a key to be able to connect to the build daemon |
|
92 |
|
and retrieve a job. So, we need to have a tool that connects |
|
93 |
|
to the daemon, get a job file, process it and post back the results. |
|
94 |
|
After a job is enqueued should we continue with hook executions? |
|
95 |
|
Should we add the notion of dependencies between jobs? |
|
96 |
|
== job is done here == |
|
97 |
|
Now, how to process the job? Especially on another machine. |
|
98 |
|
Maybe set the script also by admin and that script will finish the job? |
|
99 |
|
And doing some authentication? |
|
100 |
|
============ |
|
101 |
|
[for arm: --virt-type qemu] |
|
102 |
|
====== |
|
103 |
|
Parameters in announce: max memory, max cpus, |
|
104 |
|
I can keep the virtual machine always 'on'. Maybe revert to previous |
|
105 |
|
snapshots? |
|
106 |
|
Disable network?! |
|
107 |
|
[ ] pr: add private pull requests? |
|
108 |
|
[ ] Truncate hook output, but keep the last part. |
|
109 |
|
[ ] Add a 'label' hook. |
|
110 |
|
When another hook adds a label (for example the 'build' hook), |
|
111 |
|
trigger this hook to execute something (for example, code deploy). |
|
112 |
|
Maybe order the hooks and add a filter by label; for example: |
|
113 |
|
1. Trigger a 'build' hook => build=<status> |
|
114 |
|
2. Do code deploy if build=ok and on tag X |
|
115 |
|
[ ] Recheck token generation: I suspect I do not regenerate it on a page reload. |
|
116 |
|
[ ] Show how to limit the length of the file names in a commit. |
5 |
117 |
[ ] wh: filter by year/month/day/hour/minute/dow. |
[ ] wh: filter by year/month/day/hour/minute/dow. |
6 |
118 |
[ ] Protect emails from commits?! |
[ ] Protect emails from commits?! |
7 |
119 |
[ ] /user/catalinux/test1/source/tree/blob/"xx/"yy" on rocketgit.com |
[ ] /user/catalinux/test1/source/tree/blob/"xx/"yy" on rocketgit.com |
8 |
|
generates errors. something regargin ls-tree that outputs nothing. |
|
|
120 |
|
generates errors. Something regarding ls-tree that outputs nothing. |
9 |
121 |
This is another problem. If is empty, we should not enter foreach! |
This is another problem. If is empty, we should not enter foreach! |
10 |
122 |
[ ] mr: when pushing, also show the link to the mr? |
[ ] mr: when pushing, also show the link to the mr? |
11 |
123 |
We do not have it because we add an event. |
We do not have it because we add an event. |
File inc/builder.inc.php added (mode: 100644) (index 0000000..c5a5b88) |
|
1 |
|
<?php |
|
2 |
|
$INC = isset($INC) ? $INC : dirname(__FILE__); |
|
3 |
|
|
|
4 |
|
/* |
|
5 |
|
* Function to load job list |
|
6 |
|
*/ |
|
7 |
|
function rg_builder_load_jobs($db) |
|
8 |
|
{ |
|
9 |
|
rg_log_enter('builder_load_jobs'); |
|
10 |
|
|
|
11 |
|
$ret = array(); |
|
12 |
|
$ret['ok'] = 0; |
|
13 |
|
$ret['list'] = array(); |
|
14 |
|
|
|
15 |
|
while (1) { |
|
16 |
|
$sql = 'SELECT * FROM build_jobs' |
|
17 |
|
. ' WHERE done = 0' |
|
18 |
|
. ' ORDER BY prio DESC'; |
|
19 |
|
$res = rg_sql_query($db, $sql); |
|
20 |
|
if ($res === FALSE) |
|
21 |
|
break; |
|
22 |
|
while (($row = rg_sql_fetch_array($res))) { |
|
23 |
|
$jid = $row['id']; |
|
24 |
|
|
|
25 |
|
// TODO: set 'url' when adding the job in queue! |
|
26 |
|
$final = @unserialize($row['request']); |
|
27 |
|
if ($final === FALSE) { |
|
28 |
|
rg_internal_error('cannot unserialize!'); |
|
29 |
|
continue; |
|
30 |
|
} |
|
31 |
|
$final['id'] = $jid; |
|
32 |
|
$final['repo_id'] = $row['repo_id']; |
|
33 |
|
$final['itime'] = $row['itime']; |
|
34 |
|
$final['prio'] = $row['prio']; |
|
35 |
|
|
|
36 |
|
$ret['list'][$jid] = $final; |
|
37 |
|
} |
|
38 |
|
rg_sql_free_result($res); |
|
39 |
|
|
|
40 |
|
$ret['ok'] = 1; |
|
41 |
|
break; |
|
42 |
|
} |
|
43 |
|
|
|
44 |
|
rg_log_exit(); |
|
45 |
|
return $ret; |
|
46 |
|
} |
|
47 |
|
|
|
48 |
|
/* |
|
49 |
|
* Add a build job in queue |
|
50 |
|
*/ |
|
51 |
|
function rg_builder_add($db, $repo_id, $d) |
|
52 |
|
{ |
|
53 |
|
$ret = array('ok' => 0); |
|
54 |
|
|
|
55 |
|
rg_log_ml('builder_add: ' . print_r($d, TRUE)); |
|
56 |
|
while (1) { |
|
57 |
|
$params = array( |
|
58 |
|
'repo_id' => $repo_id, |
|
59 |
|
'prio' => 10, // TODO |
|
60 |
|
'itime' => time(), |
|
61 |
|
'request' => serialize($d) |
|
62 |
|
); |
|
63 |
|
$sql = 'INSERT INTO build_jobs (repo_id, prio, itime' |
|
64 |
|
. ', request)' |
|
65 |
|
. ' VALUES (@@repo_id@@, @@prio@@, @@itime@@' |
|
66 |
|
. ', @@request@@)'; |
|
67 |
|
$res = rg_sql_query_params($db, $sql, $params); |
|
68 |
|
if ($res === FALSE) |
|
69 |
|
break; |
|
70 |
|
|
|
71 |
|
// TODO: notify build system to not poll? |
|
72 |
|
$ret['ok'] = 1; |
|
73 |
|
break; |
|
74 |
|
} |
|
75 |
|
|
|
76 |
|
return $ret; |
|
77 |
|
} |
|
78 |
|
|
|
79 |
|
/* |
|
80 |
|
* List virtual machines |
|
81 |
|
* TODO: for now, only libvirt |
|
82 |
|
*/ |
|
83 |
|
function rg_builder_vm_list() |
|
84 |
|
{ |
|
85 |
|
$cmd = 'virsh list --name'; |
|
86 |
|
$r = rg_exec($cmd); |
|
87 |
|
if ($r['ok'] != 1) { |
|
88 |
|
rg_log('Cannot find out virtual machines: ' . $r['errmsg']); |
|
89 |
|
return FALSE; |
|
90 |
|
} |
|
91 |
|
|
|
92 |
|
return explode("\n", trim($r['data'])); |
|
93 |
|
} |
|
94 |
|
|
|
95 |
|
/* |
|
96 |
|
* Function executed when a job is done |
|
97 |
|
*/ |
|
98 |
|
function rg_builder_done($db, $job, $s) |
|
99 |
|
{ |
|
100 |
|
rg_log_enter('builder_done'); |
|
101 |
|
rg_log_ml('DEBUG: builder_done: job: ' . print_r($job, TRUE)); |
|
102 |
|
rg_log_ml('DEBUG: builder_done: status: ' . print_r($s, TRUE)); |
|
103 |
|
|
|
104 |
|
$job['done'] = time(); |
|
105 |
|
|
|
106 |
|
$ret = FALSE; |
|
107 |
|
$rollback = FALSE; |
|
108 |
|
while (1) { |
|
109 |
|
$res = rg_sql_begin($db); |
|
110 |
|
if ($res === FALSE) |
|
111 |
|
break; |
|
112 |
|
|
|
113 |
|
$rollback = TRUE; |
|
114 |
|
|
|
115 |
|
$labels = $s['labels']; |
|
116 |
|
|
|
117 |
|
// Some cosmetic stuff |
|
118 |
|
$env = $job['env']; |
|
119 |
|
$labels[] = 'worker_elap/' . ($s['done'] - $s['start']) . 's'; |
|
120 |
|
$labels[] = 'wait_time/' . ($job['worker_sent'] - $job['itime']) . 's'; |
|
121 |
|
$labels[] = 'date/' . gmdate('Y-m-d', $job['itime']); |
|
122 |
|
$labels[] = 'time/' . gmdate('H:i', $job['itime']); |
|
123 |
|
|
|
124 |
|
// add labels to the commit |
|
125 |
|
$params = array( |
|
126 |
|
'repo_id' => $job['repo_id'], |
|
127 |
|
'head' => $job['head'], |
|
128 |
|
'type' => 'build', |
|
129 |
|
'misc' => $env, |
|
130 |
|
'labels' => serialize($labels), |
|
131 |
|
'itime' => time() |
|
132 |
|
); |
|
133 |
|
$sql = 'DELETE FROM commit_labels' |
|
134 |
|
. ' WHERE repo_id = @@repo_id@@' |
|
135 |
|
. ' AND type = @@type@@' |
|
136 |
|
. ' AND misc = @@misc@@' |
|
137 |
|
. ' AND head = @@head@@'; |
|
138 |
|
$res = rg_sql_query_params($db, $sql, $params); |
|
139 |
|
if ($res === FALSE) |
|
140 |
|
break; |
|
141 |
|
rg_sql_free_result($res); |
|
142 |
|
|
|
143 |
|
$sql = 'INSERT INTO commit_labels (repo_id, head, type, misc' |
|
144 |
|
. ', labels, itime)' |
|
145 |
|
. ' VALUES (@@repo_id@@, @@head@@, @@type@@' |
|
146 |
|
. ', @@misc@@, @@labels@@, @@itime@@)'; |
|
147 |
|
$res = rg_sql_query_params($db, $sql, $params); |
|
148 |
|
if ($res === FALSE) |
|
149 |
|
break; |
|
150 |
|
rg_sql_free_result($res); |
|
151 |
|
rg_cache_unset('repo_commit_labels' . '::' . $job['repo_id'] |
|
152 |
|
. '::' . $job['head'], RG_SOCKET_NO_WAIT); |
|
153 |
|
|
|
154 |
|
$params = array( |
|
155 |
|
'id' => $job['id'], |
|
156 |
|
'done' => $job['done'], |
|
157 |
|
'status' => serialize($s) |
|
158 |
|
); |
|
159 |
|
$sql = 'UPDATE build_jobs SET done = @@done@@' |
|
160 |
|
. ', status = @@status@@' |
|
161 |
|
. ' WHERE id = @@id@@'; |
|
162 |
|
$res = rg_sql_query_params($db, $sql, $params); |
|
163 |
|
if ($res === FALSE) |
|
164 |
|
break; |
|
165 |
|
rg_sql_free_result($res); |
|
166 |
|
|
|
167 |
|
$res = rg_sql_commit($db); |
|
168 |
|
if ($res === FALSE) |
|
169 |
|
break; |
|
170 |
|
|
|
171 |
|
$rollback = FALSE; |
|
172 |
|
|
|
173 |
|
$ret = TRUE; |
|
174 |
|
break; |
|
175 |
|
} |
|
176 |
|
|
|
177 |
|
if ($rollback) |
|
178 |
|
rg_sql_rollback($db); |
|
179 |
|
|
|
180 |
|
rg_log_exit(); |
|
181 |
|
} |
|
182 |
|
|
|
183 |
|
/* |
|
184 |
|
* Returns a list of possible environments |
|
185 |
|
* TODO: think about a lot of architecures, bits, OSes! |
|
186 |
|
*/ |
|
187 |
|
function rg_builder_env_list($db) |
|
188 |
|
{ |
|
189 |
|
return array('fedora23-server-x86_64'); |
|
190 |
|
} |
|
191 |
|
|
|
192 |
|
?> |
File inc/conn.inc.php added (mode: 100644) (index 0000000..ecb8c94) |
|
1 |
|
<?php |
|
2 |
|
$INC = isset($INC) ? $INC : dirname(__FILE__); |
|
3 |
|
|
|
4 |
|
$rg_conns = array(); |
|
5 |
|
$rg_events = array('r' => array(), 'w' => array()); |
|
6 |
|
|
|
7 |
|
/* |
|
8 |
|
* Prepares a string to be sent over a socket |
|
9 |
|
*/ |
|
10 |
|
function rg_conn_prepare($s) |
|
11 |
|
{ |
|
12 |
|
if (is_array($s)) |
|
13 |
|
$s = serialize($s); |
|
14 |
|
return addcslashes($s, "\n\r\\"); |
|
15 |
|
} |
|
16 |
|
|
|
17 |
|
/* |
|
18 |
|
* Destroys a connection |
|
19 |
|
*/ |
|
20 |
|
function rg_conn_destroy($key) |
|
21 |
|
{ |
|
22 |
|
global $rg_conns; |
|
23 |
|
global $rg_events; |
|
24 |
|
|
|
25 |
|
if (!isset($rg_conns[$key])) |
|
26 |
|
rg_internal_error('key not defined!'); |
|
27 |
|
|
|
28 |
|
if ($rg_conns[$key]['exit_on_close']) |
|
29 |
|
exit(1); |
|
30 |
|
|
|
31 |
|
if (isset($rg_events['r'][$key])) |
|
32 |
|
unset($rg_events['r'][$key]); |
|
33 |
|
if (isset($rg_events['w'][$key])) |
|
34 |
|
unset($rg_events['w'][$key]); |
|
35 |
|
if (isset($rg_conns[$key]['socket'])) |
|
36 |
|
if (is_resource($rg_conns[$key]['socket'])) |
|
37 |
|
@socket_close($rg_conns[$key]['socket']); |
|
38 |
|
unset($rg_conns[$key]); |
|
39 |
|
} |
|
40 |
|
|
|
41 |
|
/* |
|
42 |
|
* Registers a new socket |
|
43 |
|
*/ |
|
44 |
|
function rg_conn_new($key, $socket) |
|
45 |
|
{ |
|
46 |
|
global $rg_conns; |
|
47 |
|
global $rg_events; |
|
48 |
|
|
|
49 |
|
$rg_conns[$key] = array( |
|
50 |
|
'socket' => $socket, |
|
51 |
|
'recv' => '', |
|
52 |
|
'send' => '', |
|
53 |
|
'itime' => time(), |
|
54 |
|
'exit_on_close' => 0, |
|
55 |
|
'func_error' => 'rg_conn_func_error', |
|
56 |
|
'func_close' => 'rg_conn_func_close' |
|
57 |
|
); |
|
58 |
|
|
|
59 |
|
$rg_events['r'][$key] = $socket; |
|
60 |
|
socket_set_nonblock($socket); |
|
61 |
|
} |
|
62 |
|
|
|
63 |
|
/* |
|
64 |
|
* Enqueues data to a socket |
|
65 |
|
*/ |
|
66 |
|
function rg_conn_enq($key, $buf) |
|
67 |
|
{ |
|
68 |
|
global $rg_conns; |
|
69 |
|
global $rg_events; |
|
70 |
|
|
|
71 |
|
$s = &$rg_conns[$key]; |
|
72 |
|
$s['send'] .= $buf; |
|
73 |
|
$rg_events['w'][$key] = $s['socket']; |
|
74 |
|
} |
|
75 |
|
|
|
76 |
|
/* |
|
77 |
|
* Called when a socket is ready to send data |
|
78 |
|
*/ |
|
79 |
|
function rg_conn_send($key) |
|
80 |
|
{ |
|
81 |
|
global $rg_conns; |
|
82 |
|
global $rg_events; |
|
83 |
|
|
|
84 |
|
$s = &$rg_conns[$key]; |
|
85 |
|
rg_log('SEND: ' . $s['send']); |
|
86 |
|
$r = @socket_send($s['socket'], $s['send'], strlen($s['send']), 0); |
|
87 |
|
if ($r === FALSE) { |
|
88 |
|
rg_log('Cannot receive!'); |
|
89 |
|
$s['func_error']($key); |
|
90 |
|
rg_conn_destroy($key); |
|
91 |
|
return FALSE; |
|
92 |
|
} |
|
93 |
|
|
|
94 |
|
$s['send'] = substr($s['send'], $r); |
|
95 |
|
if (empty($s['send'])) |
|
96 |
|
unset($rg_events['w'][$key]); |
|
97 |
|
|
|
98 |
|
return $r; |
|
99 |
|
} |
|
100 |
|
|
|
101 |
|
/* |
|
102 |
|
* Called when a socket is ready to be read |
|
103 |
|
*/ |
|
104 |
|
function rg_conn_recv($key) |
|
105 |
|
{ |
|
106 |
|
global $rg_conns; |
|
107 |
|
global $rg_events; |
|
108 |
|
|
|
109 |
|
$s = &$rg_conns[$key]; |
|
110 |
|
|
|
111 |
|
if (isset($s['func_new'])) { |
|
112 |
|
$client = @socket_accept($s['socket']); |
|
113 |
|
if ($client === FALSE) { |
|
114 |
|
rg_log('Cannot accept!'); |
|
115 |
|
return; |
|
116 |
|
} |
|
117 |
|
|
|
118 |
|
if (isset($s['func_new_arg'])) |
|
119 |
|
$arg = $s['func_new_arg']; |
|
120 |
|
else |
|
121 |
|
$arg = FALSE; |
|
122 |
|
|
|
123 |
|
socket_set_nonblock($client); |
|
124 |
|
$c = intval($client); |
|
125 |
|
rg_conn_new($c, $client); |
|
126 |
|
$s['func_new']($c, $arg); |
|
127 |
|
return; |
|
128 |
|
} |
|
129 |
|
|
|
130 |
|
$r = @socket_recv($s['socket'], $buf, 4096, 0); |
|
131 |
|
if ($r === FALSE) { |
|
132 |
|
rg_log('Cannot receive!'); |
|
133 |
|
$s['func_error']($key); |
|
134 |
|
rg_conn_destroy($key); |
|
135 |
|
return FALSE; |
|
136 |
|
} |
|
137 |
|
if ($r === 0) { |
|
138 |
|
rg_log('Cannot receive!'); |
|
139 |
|
$s['func_close']($key); |
|
140 |
|
rg_conn_destroy($key); |
|
141 |
|
return FALSE; |
|
142 |
|
} |
|
143 |
|
rg_log('RECV: ' . $buf); |
|
144 |
|
|
|
145 |
|
$s['recv'] .= $buf; |
|
146 |
|
|
|
147 |
|
if (isset($s['func_data'])) { |
|
148 |
|
$s['func_data']($key); |
|
149 |
|
return; |
|
150 |
|
} |
|
151 |
|
|
|
152 |
|
if (!isset($s['func_cmd'])) { |
|
153 |
|
rg_log_ml('key=' . $key . '; s: ' . print_r($s, TRUE)); |
|
154 |
|
rg_log('Neithter func_data nor func_cmd present!'); |
|
155 |
|
return; |
|
156 |
|
} |
|
157 |
|
|
|
158 |
|
while (1) { |
|
159 |
|
$pos = strpos($s['recv'], "\n"); |
|
160 |
|
if ($pos === FALSE) |
|
161 |
|
break; |
|
162 |
|
|
|
163 |
|
$cmd = substr($s['recv'], 0, $pos); |
|
164 |
|
$s['recv'] = substr($s['recv'], $pos + 1); |
|
165 |
|
$s['func_cmd']($key, $cmd); |
|
166 |
|
} |
|
167 |
|
} |
|
168 |
|
|
|
169 |
|
/* |
|
170 |
|
* Function that waits for activity and calls different functions |
|
171 |
|
*/ |
|
172 |
|
function rg_conn_wait($timeout) |
|
173 |
|
{ |
|
174 |
|
global $rg_conns; |
|
175 |
|
global $rg_events; |
|
176 |
|
|
|
177 |
|
$r2 = $rg_events['r']; |
|
178 |
|
$w2 = $rg_events['w']; |
|
179 |
|
$e2 = array(); |
|
180 |
|
|
|
181 |
|
$r = @socket_select($r2, $w2, $e2, $timeout); |
|
182 |
|
if ($r === FALSE) { |
|
183 |
|
rg_log('Cannot select: ' . socket_strerror(socket_last_error())); |
|
184 |
|
return; |
|
185 |
|
} |
|
186 |
|
if ($r === 0) |
|
187 |
|
return; |
|
188 |
|
|
|
189 |
|
foreach ($r2 as $key => $sock) |
|
190 |
|
rg_conn_recv($key); |
|
191 |
|
|
|
192 |
|
foreach ($w2 as $key => $sock) |
|
193 |
|
rg_conn_send($key); |
|
194 |
|
|
|
195 |
|
foreach ($e2 as $key => $sock) |
|
196 |
|
rg_conn_destroy($key); |
|
197 |
|
} |
|
198 |
|
|
|
199 |
|
/* |
|
200 |
|
* Generic function called when there is an error on a socket |
|
201 |
|
*/ |
|
202 |
|
function rg_conn_func_error($key) |
|
203 |
|
{ |
|
204 |
|
} |
|
205 |
|
|
|
206 |
|
/* |
|
207 |
|
* Generic function called when a socket closes |
|
208 |
|
*/ |
|
209 |
|
function rg_conn_func_close($key) |
|
210 |
|
{ |
|
211 |
|
} |
|
212 |
|
|
|
213 |
|
?> |
File inc/fixes.inc.php changed (mode: 100644) (index ab364f7..33eb417) |
... |
... |
include_once($INC . "/repo.inc.php"); |
12 |
12 |
include_once($INC . "/keys.inc.php"); |
include_once($INC . "/keys.inc.php"); |
13 |
13 |
|
|
14 |
14 |
$rg_fixes = array(); |
$rg_fixes = array(); |
15 |
|
$rg_fixes[1] = array("rg_fixes_user_index_by_id"); |
|
16 |
|
$rg_fixes[2] = array("rg_fixes_repo_index_by_id"); |
|
17 |
|
$rg_fixes[3] = array("rg_fixes_keys_regen"); |
|
18 |
|
$rg_fixes[4] = array("rg_fixes_repos_last_bug_id"); |
|
19 |
|
$rg_fixes[5] = array("rg_fixes_wh_ver2"); |
|
20 |
|
$rg_fixes[6] = array("rg_fixes_wh_ver3"); |
|
21 |
|
$rg_fixes[7] = array("rg_fixes_wh_ver4"); |
|
22 |
|
$rg_fixes[8] = array("rg_fixes_wh_ver5"); |
|
23 |
|
$rg_fixes[9] = array('rg_fixes_repos_last_mr_id'); |
|
|
15 |
|
$rg_fixes[1] = array( |
|
16 |
|
'functions' => 'rg_fixes_user_index_by_id' |
|
17 |
|
); |
|
18 |
|
$rg_fixes[2] = array( |
|
19 |
|
'functions' => 'rg_fixes_repo_index_by_id' |
|
20 |
|
); |
|
21 |
|
$rg_fixes[3] = array( |
|
22 |
|
'functions' => 'rg_fixes_keys_regen' |
|
23 |
|
); |
|
24 |
|
$rg_fixes[4] = array( |
|
25 |
|
'functions' => 'rg_fixes_repos_last_bug_id' |
|
26 |
|
); |
|
27 |
|
$rg_fixes[5] = array( |
|
28 |
|
'functions' => 'rg_fixes_wh_ver2' |
|
29 |
|
); |
|
30 |
|
$rg_fixes[6] = array( |
|
31 |
|
'functions' => 'rg_fixes_wh_ver3' |
|
32 |
|
); |
|
33 |
|
$rg_fixes[7] = array( |
|
34 |
|
'functions' => 'rg_fixes_wh_ver4' |
|
35 |
|
); |
|
36 |
|
$rg_fixes[8] = array( |
|
37 |
|
'functions' => 'rg_fixes_wh_ver5' |
|
38 |
|
); |
|
39 |
|
$rg_fixes[9] = array( |
|
40 |
|
'functions' => 'rg_fixes_repos_last_mr_id' |
|
41 |
|
); |
|
42 |
|
$rg_fixes[10] = array( |
|
43 |
|
'functions' => 'rg_fixes_drop_if_exists' |
|
44 |
|
); |
24 |
45 |
|
|
25 |
46 |
// This must be the last line |
// This must be the last line |
26 |
47 |
$rg_fixes_ver = count($rg_fixes); |
$rg_fixes_ver = count($rg_fixes); |
|
... |
... |
function rg_fixes_repos_last_bug_id($db) |
34 |
55 |
|
|
35 |
56 |
$ret = FALSE; |
$ret = FALSE; |
36 |
57 |
while (1) { |
while (1) { |
37 |
|
$res = rg_sql_begin($db); |
|
38 |
|
if (!$res) |
|
39 |
|
break; |
|
40 |
|
|
|
41 |
58 |
$sql = "SELECT * FROM bugs_max"; |
$sql = "SELECT * FROM bugs_max"; |
42 |
59 |
$res = rg_sql_query($db, $sql); |
$res = rg_sql_query($db, $sql); |
43 |
60 |
if (!$res) |
if (!$res) |
|
... |
... |
function rg_fixes_repos_last_bug_id($db) |
69 |
86 |
if (!$res) |
if (!$res) |
70 |
87 |
break; |
break; |
71 |
88 |
|
|
72 |
|
$res = rg_sql_commit($db); |
|
73 |
|
if (!$res) |
|
74 |
|
break; |
|
75 |
|
|
|
76 |
89 |
$ret = TRUE; |
$ret = TRUE; |
77 |
90 |
break; |
break; |
78 |
91 |
} |
} |
|
... |
... |
function rg_fixes_repos_last_mr_id($db) |
559 |
572 |
$error = FALSE; |
$error = FALSE; |
560 |
573 |
$ret = FALSE; |
$ret = FALSE; |
561 |
574 |
while (1) { |
while (1) { |
562 |
|
$res = rg_sql_begin($db); |
|
563 |
|
if (!$res) |
|
564 |
|
break; |
|
565 |
|
|
|
566 |
575 |
$sql = 'SELECT repo_id, namespace, refname' |
$sql = 'SELECT repo_id, namespace, refname' |
567 |
576 |
. ' FROM merge_requests'; |
. ' FROM merge_requests'; |
568 |
577 |
$res = rg_sql_query($db, $sql); |
$res = rg_sql_query($db, $sql); |
|
... |
... |
function rg_fixes_repos_last_mr_id($db) |
631 |
640 |
if ($error) |
if ($error) |
632 |
641 |
break; |
break; |
633 |
642 |
|
|
634 |
|
$res = rg_sql_commit($db); |
|
635 |
|
if (!$res) |
|
636 |
|
break; |
|
637 |
|
|
|
638 |
643 |
$ret = TRUE; |
$ret = TRUE; |
639 |
644 |
break; |
break; |
640 |
645 |
} |
} |
641 |
646 |
|
|
642 |
|
if ($error) |
|
643 |
|
rg_sql_rollback($db); |
|
|
647 |
|
rg_log_exit(); |
|
648 |
|
return $ret; |
|
649 |
|
} |
|
650 |
|
|
|
651 |
|
/* |
|
652 |
|
* When started to support PostgreSQL 8, 'IF EXISTS' was not working! |
|
653 |
|
*/ |
|
654 |
|
function rg_fixes_drop_if_exists($db) |
|
655 |
|
{ |
|
656 |
|
global $php_errormsg; |
|
657 |
|
|
|
658 |
|
rg_log_enter('rg_fixes_drop_if_exists'); |
|
659 |
|
|
|
660 |
|
$ret = TRUE; |
|
661 |
|
$list = array( |
|
662 |
|
'keys' => 'keys_pkey', |
|
663 |
|
'keys' => 'keys_key_key', |
|
664 |
|
'repos' => 'repos_pkey', |
|
665 |
|
'repos' => 'repos_name_key'); |
|
666 |
|
foreach ($list as $t => $c) { |
|
667 |
|
if (rg_sql_rel_exists($db, $c) != 1) |
|
668 |
|
continue; |
|
669 |
|
|
|
670 |
|
$sql = 'ALTER TABLE ' . $t . ' DROP CONSTRAINT ' . $c; |
|
671 |
|
$res = rg_sql_query($db, $sql); |
|
672 |
|
if ($res === FALSE) { |
|
673 |
|
$ret = FALSE; |
|
674 |
|
break; |
|
675 |
|
} |
|
676 |
|
rg_sql_free_result($res); |
|
677 |
|
} |
644 |
678 |
|
|
645 |
679 |
rg_log_exit(); |
rg_log_exit(); |
646 |
680 |
return $ret; |
return $ret; |
|
... |
... |
function rg_fixes_run($db, $old_ver) |
658 |
692 |
|
|
659 |
693 |
rg_log("rg_fixes_run: old_ver=$old_ver..."); |
rg_log("rg_fixes_run: old_ver=$old_ver..."); |
660 |
694 |
|
|
|
695 |
|
$ret = TRUE; |
661 |
696 |
for ($i = $old_ver + 1; $i <= $rg_fixes_ver; $i++) { |
for ($i = $old_ver + 1; $i <= $rg_fixes_ver; $i++) { |
|
697 |
|
$res = rg_sql_begin($db); |
|
698 |
|
if (!$res) { |
|
699 |
|
$ret = FALSE; |
|
700 |
|
break; |
|
701 |
|
} |
|
702 |
|
|
662 |
703 |
foreach ($rg_fixes[$i] as $function) { |
foreach ($rg_fixes[$i] as $function) { |
663 |
704 |
rg_log("Calling function $function..."); |
rg_log("Calling function $function..."); |
664 |
705 |
$r = $function($db); |
$r = $function($db); |
|
... |
... |
function rg_fixes_run($db, $old_ver) |
667 |
708 |
return FALSE; |
return FALSE; |
668 |
709 |
} |
} |
669 |
710 |
} |
} |
|
711 |
|
|
|
712 |
|
$r = rg_state_set($db, "fixes_version", $i); |
|
713 |
|
if ($r !== TRUE) { |
|
714 |
|
rg_log("Cannot set state ver (" . rg_state_error() . ")"); |
|
715 |
|
$ret = FALSE; |
|
716 |
|
break; |
|
717 |
|
} |
|
718 |
|
|
|
719 |
|
$res = rg_sql_commit($db); |
|
720 |
|
if (!$res) { |
|
721 |
|
$ret = FALSE; |
|
722 |
|
break; |
|
723 |
|
} |
670 |
724 |
} |
} |
671 |
725 |
|
|
672 |
|
return TRUE; |
|
|
726 |
|
return $ret; |
673 |
727 |
} |
} |
674 |
728 |
|
|
675 |
729 |
/* |
/* |
|
... |
... |
function rg_fixes_update($db) |
740 |
794 |
break; |
break; |
741 |
795 |
} |
} |
742 |
796 |
|
|
743 |
|
$r = rg_state_set($db, "fixes_version", $rg_fixes_ver); |
|
744 |
|
if ($r !== TRUE) { |
|
745 |
|
rg_log("Cannot set state ver (" . rg_state_error() . ")"); |
|
746 |
|
break; |
|
747 |
|
} |
|
748 |
|
|
|
749 |
797 |
$ret = TRUE; |
$ret = TRUE; |
750 |
798 |
break; |
break; |
751 |
799 |
} |
} |
File inc/git.inc.php changed (mode: 100644) (index dc57f96..6dbb650) |
... |
... |
function rg_git_log($path, $max, $from, $to, $also_patch) |
899 |
899 |
} else if (empty($_t[0])) { |
} else if (empty($_t[0])) { |
900 |
900 |
// do nothing |
// do nothing |
901 |
901 |
} else { |
} else { |
902 |
|
rg_log("DEBUG: Var [" . $_t[0] . "] has no value!"); |
|
|
902 |
|
//rg_log("DEBUG: Var [" . $_t[0] . "] has no value!"); |
903 |
903 |
} |
} |
904 |
904 |
} |
} |
905 |
905 |
|
|
|
... |
... |
function rg_git_log($path, $max, $from, $to, $also_patch) |
913 |
913 |
$y['vars']['lines_del'] = $_extra['lines_del']; |
$y['vars']['lines_del'] = $_extra['lines_del']; |
914 |
914 |
} else { |
} else { |
915 |
915 |
// stortstat |
// stortstat |
916 |
|
rg_log('DEBUG parts[1]: ' . print_r($parts[1], TRUE)); |
|
|
916 |
|
//rg_log('DEBUG parts[1]: ' . print_r($parts[1], TRUE)); |
917 |
917 |
$t = explode(',', $parts[1]); |
$t = explode(',', $parts[1]); |
918 |
918 |
|
|
919 |
919 |
for ($i = 1; $i < 3; $i++) { |
for ($i = 1; $i < 3; $i++) { |
|
... |
... |
function rg_git_log($path, $max, $from, $to, $also_patch) |
921 |
921 |
break; |
break; |
922 |
922 |
|
|
923 |
923 |
$x = trim($t[$i]); |
$x = trim($t[$i]); |
924 |
|
rg_log('DEBUG: x=[' . $x . ']'); |
|
|
924 |
|
//rg_log('DEBUG: x=[' . $x . ']'); |
925 |
925 |
if (strstr($x, 'insert')) |
if (strstr($x, 'insert')) |
926 |
926 |
$y['vars']['lines_add'] += intval($x); |
$y['vars']['lines_add'] += intval($x); |
927 |
927 |
else if (strstr($x, 'deletion')) |
else if (strstr($x, 'deletion')) |
|
... |
... |
function rg_git_log($path, $max, $from, $to, $also_patch) |
930 |
930 |
rg_log('BUG: unknown field: ' . $x); |
rg_log('BUG: unknown field: ' . $x); |
931 |
931 |
} |
} |
932 |
932 |
|
|
933 |
|
rg_log('DEBUG lines_add=' . $y['vars']['lines_add']); |
|
934 |
|
rg_log('DEBUG lines_del=' . $y['vars']['lines_del']); |
|
|
933 |
|
//rg_log('DEBUG lines_add=' . $y['vars']['lines_add']); |
|
934 |
|
//rg_log('DEBUG lines_del=' . $y['vars']['lines_del']); |
935 |
935 |
} |
} |
936 |
936 |
|
|
937 |
937 |
// final additions |
// final additions |
|
... |
... |
function rg_git_update_branch($db, $a) |
1474 |
1474 |
$ev = $a; |
$ev = $a; |
1475 |
1475 |
$ev['category'] = 3007; |
$ev['category'] = 3007; |
1476 |
1476 |
$ev['prio'] = 50; |
$ev['prio'] = 50; |
1477 |
|
$ev['ui'] = array('uid' => $a['login_uid']); |
|
|
1477 |
|
$ev['ui'] = array( |
|
1478 |
|
'organization' => $a['login_organization'], |
|
1479 |
|
'uid' => $a['login_uid'], |
|
1480 |
|
'username' => $a['login_username'] |
|
1481 |
|
); |
1478 |
1482 |
$ev['ri'] = array( |
$ev['ri'] = array( |
1479 |
1483 |
'repo_id' => $a['repo_id'], |
'repo_id' => $a['repo_id'], |
1480 |
1484 |
'name' => $a['repo_name'], |
'name' => $a['repo_name'], |
1481 |
1485 |
'url' => rg_base_url() . $a['login_url'] |
'url' => rg_base_url() . $a['login_url'] |
1482 |
|
. '/' . $a['repo_name'] |
|
|
1486 |
|
. '/' . $a['repo_name'], |
|
1487 |
|
'clone_url' => $a['repo_clone_url'] |
1483 |
1488 |
); |
); |
1484 |
1489 |
unset($ev['repo_id']); unset($ev['repo_name']); |
unset($ev['repo_id']); unset($ev['repo_name']); |
1485 |
1490 |
$r = rg_event_add($db, $ev); |
$r = rg_event_add($db, $ev); |
|
... |
... |
function rg_git_update_branch($db, $a) |
1487 |
1492 |
rg_git_fatal($a['refname'] . ": " . rg_event_error()); |
rg_git_fatal($a['refname'] . ": " . rg_event_error()); |
1488 |
1493 |
rg_event_signal_daemon('', 0); |
rg_event_signal_daemon('', 0); |
1489 |
1494 |
|
|
1490 |
|
// Here, the namespace ref is not yet updated |
|
|
1495 |
|
// TODO: Here, the namespace ref is not yet updated |
1491 |
1496 |
} |
} |
1492 |
1497 |
|
|
1493 |
1498 |
if (strcmp($a['old_rev'], $rg_git_zero) == 0) { |
if (strcmp($a['old_rev'], $rg_git_zero) == 0) { |
|
... |
... |
function rg_git_log2listing($log, $rg, $commit_table) |
1774 |
1779 |
return "Internal error"; |
return "Internal error"; |
1775 |
1780 |
$ret .= $r; |
$ret .= $r; |
1776 |
1781 |
} |
} |
|
1782 |
|
|
|
1783 |
|
if (!empty($rg['HTML:commit_labels'])) |
|
1784 |
|
$ret .= '<br />' . $rg['HTML:commit_labels']; |
1777 |
1785 |
$ret .= '</div>' . "\n"; |
$ret .= '</div>' . "\n"; |
1778 |
1786 |
|
|
1779 |
1787 |
return $ret; |
return $ret; |
File inc/repo.inc.php changed (mode: 100644) (index 56b6d4b..93f20df) |
... |
... |
function rg_repo_next_id($db, $field, $repo_id) |
104 |
104 |
*/ |
*/ |
105 |
105 |
function rg_repo_cosmetic($db, &$row) |
function rg_repo_cosmetic($db, &$row) |
106 |
106 |
{ |
{ |
|
107 |
|
global $rg_ssh_port, $rg_git_port; |
|
108 |
|
|
|
109 |
|
$row['clone_url'] = ''; |
|
110 |
|
|
107 |
111 |
// Because this field was added later and the cache may not have it |
// Because this field was added later and the cache may not have it |
108 |
112 |
if (!isset($row['template'])) |
if (!isset($row['template'])) |
109 |
113 |
$row['template'] = ''; |
$row['template'] = ''; |
|
... |
... |
function rg_repo_cosmetic($db, &$row) |
123 |
127 |
$row['owner'] = $_ui['username']; |
$row['owner'] = $_ui['username']; |
124 |
128 |
$row['url_user'] = rg_base_url() . rg_re_userpage($_ui); |
$row['url_user'] = rg_base_url() . rg_re_userpage($_ui); |
125 |
129 |
$row['url_repo'] = rg_base_url() . rg_re_repopage($_ui, $row['name']); |
$row['url_repo'] = rg_base_url() . rg_re_repopage($_ui, $row['name']); |
|
130 |
|
|
|
131 |
|
$row['clone_url_ssh'] = rg_re_repo_ssh($_ui['organization'], |
|
132 |
|
$_ui['username'], $row['name']); |
|
133 |
|
$row['clone_url_git'] = rg_re_repo_git($_ui['organization'], |
|
134 |
|
$_ui['username'], $row['name']); |
|
135 |
|
if ($rg_ssh_port != 0) |
|
136 |
|
$row['clone_url'] = $row['clone_url_ssh']; |
|
137 |
|
else |
|
138 |
|
$row['clone_url'] = $row['clone_url_git']; |
126 |
139 |
} |
} |
127 |
140 |
|
|
128 |
141 |
$master_repo = '-'; |
$master_repo = '-'; |
|
... |
... |
function rg_repo_event_new($db, $event) |
483 |
496 |
|
|
484 |
497 |
// webhook |
// webhook |
485 |
498 |
$x = $event; |
$x = $event; |
486 |
|
$x['category'] = 30000; |
|
|
499 |
|
$x['category'] = 'wh_send'; |
487 |
500 |
$x['prio'] = 50; |
$x['prio'] = 50; |
488 |
501 |
$x['wh_event'] = 'C'; // see rg_wh_events array |
$x['wh_event'] = 'C'; // see rg_wh_events array |
489 |
502 |
$ret[] = $x; |
$ret[] = $x; |
|
... |
... |
function rg_repo_event_push($db, $event) |
740 |
753 |
|
|
741 |
754 |
// webhook |
// webhook |
742 |
755 |
$x = $event; |
$x = $event; |
743 |
|
$x['category'] = 30000; |
|
|
756 |
|
$x['category'] = 'wh_send'; |
744 |
757 |
$x['wh_event'] = 'P'; // push |
$x['wh_event'] = 'P'; // push |
745 |
758 |
$ret[] = $x; |
$ret[] = $x; |
746 |
759 |
|
|
|
... |
... |
function rg_repo_stats($rg) |
2148 |
2161 |
return $ret; |
return $ret; |
2149 |
2162 |
} |
} |
2150 |
2163 |
|
|
|
2164 |
|
/*************************** commit_labels stuff */ |
|
2165 |
|
|
|
2166 |
|
/* |
|
2167 |
|
* Returns a list of labels attached to a commit |
|
2168 |
|
*/ |
|
2169 |
|
function rg_repo_commit_labels($db, $repo_id, $head) |
|
2170 |
|
{ |
|
2171 |
|
rg_prof_start('repo_commit_labels'); |
|
2172 |
|
rg_log_enter('repo_commit_labels: repo_id=' |
|
2173 |
|
. $repo_id . ' head=' . $head); |
|
2174 |
|
|
|
2175 |
|
$ret = array(); |
|
2176 |
|
$ret['ok'] = 0; |
|
2177 |
|
while(1) { |
|
2178 |
|
$key = 'repo_commit_labels' . '::' . $repo_id . '::' . $head; |
|
2179 |
|
$c = rg_cache_get($key); |
|
2180 |
|
if ($c !== FALSE) { |
|
2181 |
|
$ret = $c; |
|
2182 |
|
//rg_repo_cosmetic($db, $ret); |
|
2183 |
|
break; |
|
2184 |
|
} |
|
2185 |
|
|
|
2186 |
|
$params = array( |
|
2187 |
|
'repo_id' => $repo_id, |
|
2188 |
|
'head' => $head |
|
2189 |
|
); |
|
2190 |
|
$sql = 'SELECT * FROM commit_labels' |
|
2191 |
|
. ' WHERE repo_id = @@repo_id@@' |
|
2192 |
|
. ' AND head = @@head@@'; |
|
2193 |
|
$res = rg_sql_query_params($db, $sql, $params); |
|
2194 |
|
if ($res === FALSE) { |
|
2195 |
|
rg_repo_set_error('cannot query'); |
|
2196 |
|
break; |
|
2197 |
|
} |
|
2198 |
|
$rows = rg_sql_num_rows($res); |
|
2199 |
|
$ret['list'] = array(); |
|
2200 |
|
while (($row = rg_sql_fetch_array($res))) { |
|
2201 |
|
// Cosmetic |
|
2202 |
|
$row['labels'] = @unserialize($row['labels']); |
|
2203 |
|
$row['itime_nice'] = gmdate('Y-m-d H:i', $row['itime']); |
|
2204 |
|
$ret['list'][] = $row; |
|
2205 |
|
} |
|
2206 |
|
rg_sql_free_result($res); |
|
2207 |
|
|
|
2208 |
|
$ret['ok'] = 1; |
|
2209 |
|
|
|
2210 |
|
rg_cache_set($key, $ret, RG_SOCKET_NO_WAIT); |
|
2211 |
|
break; |
|
2212 |
|
} |
|
2213 |
|
|
|
2214 |
|
//rg_log_ml('DEBUG: commit_labels: ' . print_r($ret, TRUE)); |
|
2215 |
|
|
|
2216 |
|
rg_log_exit(); |
|
2217 |
|
rg_prof_end('repo_commit_labels'); |
|
2218 |
|
return $ret; |
|
2219 |
|
} |
|
2220 |
|
|
|
2221 |
|
/* |
|
2222 |
|
* Show commit labels as nice html |
|
2223 |
|
*/ |
|
2224 |
|
function rg_repo_commit_labels_html($db, $repo_id, $commit) |
|
2225 |
|
{ |
|
2226 |
|
rg_log_enter('repo_commit_labels_html'); |
|
2227 |
|
|
|
2228 |
|
$ret = ''; |
|
2229 |
|
while (1) { |
|
2230 |
|
if (empty($commit)) |
|
2231 |
|
break; |
|
2232 |
|
|
|
2233 |
|
$rg = array(); |
|
2234 |
|
|
|
2235 |
|
$r = rg_repo_commit_labels($db, $repo_id, $commit); |
|
2236 |
|
//rg_log_ml('DEBUG: r: ' . print_r($r, TRUE)); |
|
2237 |
|
if ($r['ok'] != 1) { |
|
2238 |
|
$ret = rg_template('repo/cl/ierror.html', |
|
2239 |
|
$rg, TRUE /*xss*/); |
|
2240 |
|
break; |
|
2241 |
|
} |
|
2242 |
|
|
|
2243 |
|
if (empty($r['list'])) |
|
2244 |
|
break; |
|
2245 |
|
|
|
2246 |
|
foreach ($r['list'] as $index => &$i) |
|
2247 |
|
$i['labels_nice'] = implode(' ', $i['labels']); |
|
2248 |
|
|
|
2249 |
|
$ret = rg_template_table('repo/cl/list', $r['list'], $rg); |
|
2250 |
|
break; |
|
2251 |
|
} |
|
2252 |
|
|
|
2253 |
|
rg_log_exit(); |
|
2254 |
|
return $ret; |
|
2255 |
|
} |
|
2256 |
|
|
2151 |
2257 |
?> |
?> |
File inc/wh/build.inc.php added (mode: 100644) (index 0000000..f694916) |
|
1 |
|
<?php |
|
2 |
|
require_once($INC . "/util.inc.php"); |
|
3 |
|
require_once($INC . "/log.inc.php"); |
|
4 |
|
require_once($INC . "/sql.inc.php"); |
|
5 |
|
require_once($INC . "/prof.inc.php"); |
|
6 |
|
require_once($INC . "/events.inc.php"); |
|
7 |
|
require_once($INC . '/builder.inc.php'); |
|
8 |
|
require_once($INC . "/wh/core.inc.php"); |
|
9 |
|
|
|
10 |
|
$rg_wh_build_functions = array( |
|
11 |
|
'wh_build_send' => 'rg_wh_build_send', |
|
12 |
|
'wh_build_send_one' => 'rg_wh_build_send_one' |
|
13 |
|
); |
|
14 |
|
rg_event_register_functions($rg_wh_build_functions); |
|
15 |
|
|
|
16 |
|
|
|
17 |
|
/* |
|
18 |
|
* Helper for rg_wh_build_send |
|
19 |
|
*/ |
|
20 |
|
function rg_wh_build_send_one($db, $event) |
|
21 |
|
{ |
|
22 |
|
rg_prof_start('wh_build_send_one'); |
|
23 |
|
rg_log_ml('wh_build_send_one: event: ' . print_r($event, TRUE)); |
|
24 |
|
|
|
25 |
|
$ret = FALSE; |
|
26 |
|
|
|
27 |
|
$wh = &$event['wh']; |
|
28 |
|
|
|
29 |
|
$last_output = ''; |
|
30 |
|
while (1) { |
|
31 |
|
// replace ##tags## |
|
32 |
|
rg_wh_replace_tags($event); |
|
33 |
|
|
|
34 |
|
// we need to copy 'flags'/'url' because we pass idata |
|
35 |
|
$wh['idata']['flags'] = $wh['flags']; |
|
36 |
|
$wh['idata']['url'] = $event['ri']['clone_url']; |
|
37 |
|
$wh['idata']['head'] = $event['new_rev']; |
|
38 |
|
|
|
39 |
|
// TODO: still have to pass env |
|
40 |
|
// TODO: also, e-mail notification |
|
41 |
|
$wh['idata']['env'] = 'fedora23-server-x86_64'; |
|
42 |
|
|
|
43 |
|
// Call the function |
|
44 |
|
$r = rg_builder_add($db, $event['ri']['repo_id'], $wh['idata']); |
|
45 |
|
if ($r['ok'] != 1) { |
|
46 |
|
$last_output .= $r['error']; |
|
47 |
|
break; |
|
48 |
|
} |
|
49 |
|
|
|
50 |
|
$ret = array(); |
|
51 |
|
break; |
|
52 |
|
} |
|
53 |
|
|
|
54 |
|
rg_wh_set_last_output($db, $event['ui']['uid'], $wh['id'], |
|
55 |
|
substr($last_output, 0, 4096)); |
|
56 |
|
|
|
57 |
|
rg_prof_end('wh_build_send_one'); |
|
58 |
|
return $ret; |
|
59 |
|
} |
|
60 |
|
|
|
61 |
|
/* |
|
62 |
|
* Generic function which will be called when a webhook must be posted |
|
63 |
|
*/ |
|
64 |
|
function rg_wh_build_send($db, $event) |
|
65 |
|
{ |
|
66 |
|
rg_prof_start('wh_build_send'); |
|
67 |
|
rg_log_ml('wh_build_send: event: ' . print_r($event, TRUE)); |
|
68 |
|
|
|
69 |
|
$ret = array(); |
|
70 |
|
|
|
71 |
|
// First, get the list of hooks |
|
72 |
|
$r = rg_wh_list($db, $event['ui']['uid']); |
|
73 |
|
if ($r['ok'] != 1) |
|
74 |
|
return FALSE; |
|
75 |
|
|
|
76 |
|
// Filter |
|
77 |
|
foreach ($r['list'] as $id => $wh) { |
|
78 |
|
if (strcmp($wh['htype'], 'build') != 0) |
|
79 |
|
continue; |
|
80 |
|
|
|
81 |
|
// Diabled? |
|
82 |
|
if (strchr($wh['flags'], 'D')) |
|
83 |
|
continue; |
|
84 |
|
|
|
85 |
|
if (isset($event['ri']['name']) |
|
86 |
|
&& !rg_repo_compare_refs($wh['repo'], $event['ri']['name'])) { |
|
87 |
|
rg_log('hook is not for this repo'); |
|
88 |
|
continue; |
|
89 |
|
} |
|
90 |
|
|
|
91 |
|
if (isset($event['refname']) |
|
92 |
|
&& !rg_repo_compare_refs($wh['refname'], $event['refname'])) { |
|
93 |
|
rg_log('hook is not for this ref'); |
|
94 |
|
continue; |
|
95 |
|
} |
|
96 |
|
|
|
97 |
|
$x = $event; |
|
98 |
|
$x['category'] = 'wh_build_send_one'; |
|
99 |
|
$x['wh'] = $wh; |
|
100 |
|
$x['debug'] = $wh['idata']['debug']; |
|
101 |
|
$ret[] = $x; |
|
102 |
|
} |
|
103 |
|
|
|
104 |
|
rg_prof_end('wh_build_send'); |
|
105 |
|
return $ret; |
|
106 |
|
} |
|
107 |
|
|
|
108 |
|
/* |
|
109 |
|
* Some cosmetics applied to a webhook |
|
110 |
|
*/ |
|
111 |
|
function rg_wh_build_cosmetic(&$row) |
|
112 |
|
{ |
|
113 |
|
$f = rg_template('user/settings/wh/build/show_one.html', |
|
114 |
|
$rg, TRUE/*xss*/); |
|
115 |
|
|
|
116 |
|
$list = ''; |
|
117 |
|
foreach ($row['idata']['cmds'] as $i => &$info) { |
|
118 |
|
if (empty($info['cmd'])) |
|
119 |
|
continue; |
|
120 |
|
$list .= str_replace('##i##', $i, $f); |
|
121 |
|
|
|
122 |
|
$info['abort_nice'] = |
|
123 |
|
$info['abort'] == 1 ? "Yes" : "No"; |
|
124 |
|
} |
|
125 |
|
$row['idata']['HTML:wh_list'] = |
|
126 |
|
rg_template_string($list, 0 /*off*/, $row['idata'], TRUE /*xss*/); |
|
127 |
|
|
|
128 |
|
$row['idata']['HTML:private'] = rg_template( |
|
129 |
|
'user/settings/wh/build/show.html', $row['idata'], TRUE /*xss*/); |
|
130 |
|
} |
|
131 |
|
|
|
132 |
|
/* |
|
133 |
|
* Fill private data based on parameters passed |
|
134 |
|
*/ |
|
135 |
|
function rg_wh_build_fill_vars(&$rg) |
|
136 |
|
{ |
|
137 |
|
$a = &$rg['wh']['idata']; |
|
138 |
|
|
|
139 |
|
$a['cmds'] = array(); |
|
140 |
|
for ($i = 1; $i <= 5; $i++) { |
|
141 |
|
$p = 'wh::idata::cmds::' . $i . '::'; |
|
142 |
|
|
|
143 |
|
$cmd = trim(rg_var_str($p . 'cmd')); |
|
144 |
|
if (empty($cmd)) { |
|
145 |
|
$a['cmds'][$i] = array( |
|
146 |
|
'cmd' => '', |
|
147 |
|
'label_ok' => '', |
|
148 |
|
'label_nok' => '', |
|
149 |
|
'abort' => 0 |
|
150 |
|
); |
|
151 |
|
continue; |
|
152 |
|
} |
|
153 |
|
|
|
154 |
|
$a['cmds'][$i] = array(); |
|
155 |
|
$a['cmds'][$i]['cmd'] = $cmd; |
|
156 |
|
$a['cmds'][$i]['label_ok'] = trim(rg_var_str($p . 'label_ok')); |
|
157 |
|
$a['cmds'][$i]['label_nok'] = trim(rg_var_str($p . 'label_nok')); |
|
158 |
|
$x = trim(rg_var_str($p . 'abort')); |
|
159 |
|
$a['cmds'][$i]['abort'] = strcasecmp($x, 'on') == 0 ? 1 : 0; |
|
160 |
|
} |
|
161 |
|
|
|
162 |
|
//rg_log_ml('DEBUG: after fill_vars: ' . print_r($a, TRUE)); |
|
163 |
|
} |
|
164 |
|
|
|
165 |
|
/* |
|
166 |
|
* Validate parameters passed |
|
167 |
|
*/ |
|
168 |
|
function rg_wh_build_validate_vars($rg, &$errmsg) |
|
169 |
|
{ |
|
170 |
|
global $rg_wh_build_itypes; |
|
171 |
|
|
|
172 |
|
$a = $rg['wh']; |
|
173 |
|
|
|
174 |
|
$ret = FALSE; |
|
175 |
|
while (1) { |
|
176 |
|
$all_empty = TRUE; |
|
177 |
|
|
|
178 |
|
rg_log_ml('DEBUG: cmds:' . print_r($a['idata']['cmds'], TRUE)); |
|
179 |
|
if (empty($a['idata']['cmds'])) { |
|
180 |
|
$errmsg[] = rg_template('user/settings/wh/build/inv_cmd.txt', |
|
181 |
|
$rg, TRUE /*xss*/); |
|
182 |
|
break; |
|
183 |
|
} |
|
184 |
|
|
|
185 |
|
$ret = TRUE; |
|
186 |
|
break; |
|
187 |
|
} |
|
188 |
|
|
|
189 |
|
return $ret; |
|
190 |
|
} |
|
191 |
|
|
|
192 |
|
/* |
|
193 |
|
* Transfers to $rg the custom parameters - used when showing the form |
|
194 |
|
*/ |
|
195 |
|
function rg_wh_build_add_form(&$rg) |
|
196 |
|
{ |
|
197 |
|
$f = rg_template_blind('user/settings/wh/build/form_cmd.html'); |
|
198 |
|
|
|
199 |
|
$cmds = ''; |
|
200 |
|
for ($i = 1; $i <= 5; $i++) |
|
201 |
|
$cmds .= str_replace('##i##', $i, $f); |
|
202 |
|
|
|
203 |
|
//rg_log_ml('DEBUG: cmds=' . $cmds); |
|
204 |
|
|
|
205 |
|
$rg['HTML:cmds'] = |
|
206 |
|
rg_template_string($cmds, 0 /*off*/, $rg, TRUE /*xss*/); |
|
207 |
|
|
|
208 |
|
$rg['HTML:custom_form'] = rg_template('user/settings/wh/build/form.html', |
|
209 |
|
$rg, TRUE /*xss*/); |
|
210 |
|
} |
|
211 |
|
|
|
212 |
|
/* |
|
213 |
|
* Add custom hints |
|
214 |
|
*/ |
|
215 |
|
function rg_wh_build_fill_hints($rg, &$hints) |
|
216 |
|
{ |
|
217 |
|
$hints[]['HTML:hint'] = rg_template('user/settings/wh/build/hints.html', |
|
218 |
|
$rg, TRUE /*xss*/); |
|
219 |
|
} |
|
220 |
|
|
|
221 |
|
/* |
|
222 |
|
* Loads default paras for a form |
|
223 |
|
*/ |
|
224 |
|
function rg_wh_build_default_paras(&$rg) |
|
225 |
|
{ |
|
226 |
|
$a = &$rg['wh']['idata']; |
|
227 |
|
|
|
228 |
|
$t = array( |
|
229 |
|
'cmd' => '', |
|
230 |
|
'label_ok' => '', |
|
231 |
|
'label_nok' => '', |
|
232 |
|
'abort' => 0 |
|
233 |
|
); |
|
234 |
|
|
|
235 |
|
for ($i = 1; $i <= 5; $i++) |
|
236 |
|
$a['cmds'][$i] = $t; |
|
237 |
|
|
|
238 |
|
$a['events'] = 'P'; |
|
239 |
|
} |
|
240 |
|
|
|
241 |
|
|
|
242 |
|
|
|
243 |
|
$rg_wh_plugins['build'] = array( |
|
244 |
|
'htype' => 'build', |
|
245 |
|
'description' => 'Clone, builds and test', |
|
246 |
|
'cosmetic' => 'rg_wh_build_cosmetic', |
|
247 |
|
'fill_vars' => 'rg_wh_build_fill_vars', |
|
248 |
|
'validate_vars' => 'rg_wh_build_validate_vars', |
|
249 |
|
'add_form' => 'rg_wh_build_add_form', |
|
250 |
|
'default_paras' => 'rg_wh_build_default_paras', |
|
251 |
|
'fill_hints' => 'rg_wh_build_fill_hints', |
|
252 |
|
'have_events' => TRUE, |
|
253 |
|
'flags' => array() |
|
254 |
|
); |
|
255 |
|
|
|
256 |
|
?> |
File rocketgit.spec.in changed (mode: 100644) (index 04c0bd1..3db0c91) |
... |
... |
Release: @REV@ |
10 |
10 |
License: Affero GPLv3+ |
License: Affero GPLv3+ |
11 |
11 |
Group: Development/Tools |
Group: Development/Tools |
12 |
12 |
Source: http://kernel.embedromix.ro/us/rocketgit/%{name}-%{version}.tar.gz |
Source: http://kernel.embedromix.ro/us/rocketgit/%{name}-%{version}.tar.gz |
13 |
|
URL: http://kernel.embedromix.ro/us/ |
|
|
13 |
|
URL: https://rocketgit.com/ |
14 |
14 |
BuildRoot: %{_tmppath}/%{name}-%{version}-buildroot |
BuildRoot: %{_tmppath}/%{name}-%{version}-buildroot |
15 |
15 |
BuildArch: noarch |
BuildArch: noarch |
16 |
16 |
Requires: httpd, mod_ssl, php, php-cli, php-pgsql, php-mbstring, xinetd |
Requires: httpd, mod_ssl, php, php-cli, php-pgsql, php-mbstring, xinetd |
17 |
|
Requires: git, cronie |
|
|
17 |
|
Requires: git, cronie, postgresql-server |
18 |
18 |
Requires: util-linux |
Requires: util-linux |
19 |
19 |
# SELinux stuff |
# SELinux stuff |
20 |
20 |
# https://fedoraproject.org/wiki/SELinux_Policy_Modules_Packaging_Draft?rd=PackagingDrafts/SELinux/PolicyModules |
# https://fedoraproject.org/wiki/SELinux_Policy_Modules_Packaging_Draft?rd=PackagingDrafts/SELinux/PolicyModules |
|
... |
... |
rm -rf ${RPM_BUILD_ROOT} |
90 |
90 |
%config(noreplace) @ETC@/httpd/conf.d/rocketgit.conf |
%config(noreplace) @ETC@/httpd/conf.d/rocketgit.conf |
91 |
91 |
%attr(0755,rocketgit,rocketgit) %dir @VAR_LOG@/@PRJ@ |
%attr(0755,rocketgit,rocketgit) %dir @VAR_LOG@/@PRJ@ |
92 |
92 |
%attr(0755,apache,apache) %dir @VAR_LOG@/@PRJ@-web |
%attr(0755,apache,apache) %dir @VAR_LOG@/@PRJ@-web |
93 |
|
%attr(0755,rocketgit,rocketgit) %dir @VAR_LIB@/@PRJ@ |
|
|
93 |
|
%attr(0755,root,root) %dir @VAR_LIB@/@PRJ@ |
94 |
94 |
%attr(0755,rocketgit,rocketgit) %dir @VAR_LIB@/@PRJ@/locks |
%attr(0755,rocketgit,rocketgit) %dir @VAR_LIB@/@PRJ@/locks |
95 |
95 |
%attr(0755,rocketgit,rocketgit) %dir @VAR_LIB@/@PRJ@/repos |
%attr(0755,rocketgit,rocketgit) %dir @VAR_LIB@/@PRJ@/repos |
96 |
96 |
%attr(0755,rocketgit,rocketgit) %dir @VAR_LIB@/@PRJ@/q_merge_requests |
%attr(0755,rocketgit,rocketgit) %dir @VAR_LIB@/@PRJ@/q_merge_requests |
97 |
97 |
%attr(0755,rocketgit,rocketgit) %dir @VAR_LIB@/@PRJ@/sockets |
%attr(0755,rocketgit,rocketgit) %dir @VAR_LIB@/@PRJ@/sockets |
98 |
98 |
%attr(0700,rocketgit,rocketgit) %dir @VAR_LIB@/@PRJ@/tmp |
%attr(0700,rocketgit,rocketgit) %dir @VAR_LIB@/@PRJ@/tmp |
|
99 |
|
%attr(0700,root,root) %dir @VAR_LIB@/@PRJ@/worker |
99 |
100 |
%attr(0700,root,root) @USR_SBIN@/* |
%attr(0700,root,root) @USR_SBIN@/* |
100 |
101 |
@USR_SHARE@/@PRJ@/* |
@USR_SHARE@/@PRJ@/* |
101 |
102 |
@USR_SHARE@/selinux/*/@PRJ@.pp |
@USR_SHARE@/selinux/*/@PRJ@.pp |
File scripts/builder.php added (mode: 100644) (index 0000000..250ef08) |
|
1 |
|
<?php |
|
2 |
|
// This is called by cron, and is persistent. |
|
3 |
|
// It takes care of build jobs. |
|
4 |
|
error_reporting(E_ALL); |
|
5 |
|
ini_set("track_errors", "On"); |
|
6 |
|
set_time_limit(0); |
|
7 |
|
|
|
8 |
|
$_s = microtime(TRUE); |
|
9 |
|
|
|
10 |
|
require_once("/etc/rocketgit/config.php"); |
|
11 |
|
|
|
12 |
|
$INC = dirname(__FILE__) . "/../inc"; |
|
13 |
|
require_once($INC . "/init.inc.php"); |
|
14 |
|
require_once($INC . "/log.inc.php"); |
|
15 |
|
require_once($INC . "/sql.inc.php"); |
|
16 |
|
require_once($INC . "/struct.inc.php"); |
|
17 |
|
require_once($INC . "/events.inc.php"); |
|
18 |
|
require_once($INC . "/repo.inc.php"); |
|
19 |
|
require_once($INC . "/prof.inc.php"); |
|
20 |
|
require_once($INC . "/mr.inc.php"); |
|
21 |
|
require_once($INC . "/keys.inc.php"); |
|
22 |
|
require_once($INC . "/user.inc.php"); |
|
23 |
|
require_once($INC . "/bug.inc.php"); |
|
24 |
|
require_once($INC . "/fixes.inc.php"); |
|
25 |
|
require_once($INC . "/plan.inc.php"); |
|
26 |
|
require_once($INC . "/admin.inc.php"); |
|
27 |
|
require_once($INC . "/ver.php"); |
|
28 |
|
require_once($INC . "/builder.inc.php"); |
|
29 |
|
require_once($INC . "/conn.inc.php"); |
|
30 |
|
|
|
31 |
|
/* |
|
32 |
|
* Called when a new client connects |
|
33 |
|
*/ |
|
34 |
|
function xnew($key, $arg) |
|
35 |
|
{ |
|
36 |
|
global $rg_conns; |
|
37 |
|
|
|
38 |
|
$s = &$rg_conns[$key]; |
|
39 |
|
$s['func_cmd'] = 'xdispatch'; |
|
40 |
|
$s['db'] = $arg; |
|
41 |
|
$s['auth'] = 0; |
|
42 |
|
} |
|
43 |
|
|
|
44 |
|
function xdispatch($key, $line) |
|
45 |
|
{ |
|
46 |
|
global $rg_conns; |
|
47 |
|
global $jobs; |
|
48 |
|
global $rg_builder_key; |
|
49 |
|
|
|
50 |
|
rg_log('Dispatch[' . $key . ']'); |
|
51 |
|
|
|
52 |
|
$s = &$rg_conns[$key]; |
|
53 |
|
|
|
54 |
|
$cmd = substr($line, 0, 4); |
|
55 |
|
$d = trim(substr($line, 4)); |
|
56 |
|
$x = stripcslashes($d); |
|
57 |
|
$u = @unserialize($x); |
|
58 |
|
if ($u === FALSE) { |
|
59 |
|
rg_conn_destroy($key); |
|
60 |
|
return; |
|
61 |
|
} |
|
62 |
|
|
|
63 |
|
rg_log_ml('cmd=' . $cmd . ' u: ' . print_r($u, TRUE)); |
|
64 |
|
|
|
65 |
|
if (strcmp($cmd, 'ANN ') == 0) { |
|
66 |
|
$now = time(); |
|
67 |
|
if ($u['boot_time'] < $now - 10) { |
|
68 |
|
rg_log('boot_time is too old; abort'); |
|
69 |
|
rg_conn_destroy($key); |
|
70 |
|
return; |
|
71 |
|
} |
|
72 |
|
$sign = hash_hmac('sha512', $u['boot_time'], $rg_builder_key); |
|
73 |
|
if (strcmp($sign, $u['sign']) != 0) { |
|
74 |
|
rg_log('signature is not ok [' . $sign . ']' |
|
75 |
|
. ' != [' . $u['sign'] . ']'); |
|
76 |
|
rg_conn_destroy($key); |
|
77 |
|
return; |
|
78 |
|
} |
|
79 |
|
$s['ann'] = $u; |
|
80 |
|
$s['auth'] = 1; |
|
81 |
|
rg_log('Peer announced itself!'); |
|
82 |
|
} else if (strcmp($cmd, 'STA ') == 0) { |
|
83 |
|
if ($s['auth'] != 1) { |
|
84 |
|
rg_log($key . ':Client not authenticated!'); |
|
85 |
|
$a = array('error' => 'client not authenticated'); |
|
86 |
|
$cmd = 'ERR ' . rg_conn_prepare($a) . "\n"; |
|
87 |
|
rg_conn_enq($key, $cmd); |
|
88 |
|
return; |
|
89 |
|
} |
|
90 |
|
$jid = $u['id']; |
|
91 |
|
$jobs[$jid]['worker'] = $key; |
|
92 |
|
$jobs[$jid]['worker_started'] = time(); |
|
93 |
|
rg_log('Job started: ' . $jid); |
|
94 |
|
} else if (strcmp($cmd, 'DON ') == 0) { |
|
95 |
|
if ($s['auth'] != 1) { |
|
96 |
|
rg_log($key . ':Client not authenticated!'); |
|
97 |
|
$a = array('error' => 'client not authenticated'); |
|
98 |
|
$cmd = 'ERR ' . rg_conn_prepare($a) . "\n"; |
|
99 |
|
rg_conn_enq($key, $cmd); |
|
100 |
|
return; |
|
101 |
|
} |
|
102 |
|
$jid = $u['id']; |
|
103 |
|
$r = rg_builder_done($s['db'], $jobs[$jid], $u['status']); |
|
104 |
|
if ($r === TRUE) { |
|
105 |
|
unset($jobs[$jid]); |
|
106 |
|
// Send DoneREceived - so client will delete the jobs |
|
107 |
|
$a = array('id' => $job['id']); |
|
108 |
|
$cmd = 'DRE ' . rg_conn_prepare($a) . "\n"; |
|
109 |
|
rg_conn_enq($key, $cmd); |
|
110 |
|
} |
|
111 |
|
} else { |
|
112 |
|
rg_log('Unknown command [' . $cmd . ']!'); |
|
113 |
|
} |
|
114 |
|
} |
|
115 |
|
|
|
116 |
|
function rg_process_queue($db, &$job) |
|
117 |
|
{ |
|
118 |
|
global $rg_conns; |
|
119 |
|
|
|
120 |
|
rg_log('process_queue: worker=' . $job['worker']); |
|
121 |
|
if (!empty($job['worker'])) |
|
122 |
|
return; |
|
123 |
|
|
|
124 |
|
rg_log_ml('Processing job: ' . print_r($job, TRUE)); |
|
125 |
|
|
|
126 |
|
// Find a target for this job |
|
127 |
|
$found = FALSE; |
|
128 |
|
foreach ($rg_conns as $key => $i) { |
|
129 |
|
// Is it a worker? |
|
130 |
|
if (!isset($i['ann'])) { |
|
131 |
|
rg_log('Conn ' . $key . ' has no announce.'); |
|
132 |
|
// TODO: close after some time? |
|
133 |
|
continue; |
|
134 |
|
} |
|
135 |
|
|
|
136 |
|
if (empty($i['ann']['env'])) { |
|
137 |
|
rg_log('Conn ' . $key . ' has no environments.'); |
|
138 |
|
continue; |
|
139 |
|
} |
|
140 |
|
|
|
141 |
|
foreach ($i['ann']['env'] as $env => $junk) { |
|
142 |
|
if (strcmp($job['env'], $env) != 0) { |
|
143 |
|
rg_log('job env [' . $job['env'] . ']' |
|
144 |
|
. ' != worker [' . $env . ']'); |
|
145 |
|
continue; |
|
146 |
|
} |
|
147 |
|
|
|
148 |
|
// Remove some stuff |
|
149 |
|
$job2 = $job; |
|
150 |
|
unset($job2['worker']); |
|
151 |
|
unset($job2['events']); |
|
152 |
|
unset($job2['flags']); |
|
153 |
|
|
|
154 |
|
$cmd = 'BLD ' . rg_conn_prepare($job2) . "\n"; |
|
155 |
|
rg_conn_enq($key, $cmd); |
|
156 |
|
// TODO: get a confirmation? |
|
157 |
|
$job['worker'] = $key; |
|
158 |
|
$job['worker_started'] = 0; |
|
159 |
|
$job['worker_sent'] = time(); |
|
160 |
|
rg_log_ml('After sending BLD: ' . print_r($job, TRUE)); |
|
161 |
|
// TODO: after some time, if worker_started is still 0, |
|
162 |
|
// mark the 'worker' as '' to be able to go in other place |
|
163 |
|
// TODO: maybe the client must resync with server to |
|
164 |
|
// abort jobs already done on another host, to not |
|
165 |
|
// duplicate work |
|
166 |
|
$found = TRUE; |
|
167 |
|
return; |
|
168 |
|
} |
|
169 |
|
} |
|
170 |
|
|
|
171 |
|
if (!$found) |
|
172 |
|
rg_log('No worker found!'); |
|
173 |
|
} |
|
174 |
|
|
|
175 |
|
|
|
176 |
|
rg_prof_start("MAIN"); |
|
177 |
|
|
|
178 |
|
rg_log_set_file($rg_log_dir . "/builder.log"); |
|
179 |
|
rg_log_set_sid("000000"); // to spread the logs |
|
180 |
|
// We must disable cache, else, we will not receive the updates because |
|
181 |
|
// of the core cache. Do not forget that we are a long live process. |
|
182 |
|
$rg_cache_core_enable = FALSE; |
|
183 |
|
|
|
184 |
|
rg_log("Start (ver=$rocketgit_version)..."); |
|
185 |
|
|
|
186 |
|
rg_sql_app("rg-builder"); |
|
187 |
|
$db = rg_sql_open($rg_sql); |
|
188 |
|
if ($db === FALSE) { |
|
189 |
|
rg_internal_error("Cannot connect to database!"); |
|
190 |
|
exit(1); |
|
191 |
|
} |
|
192 |
|
|
|
193 |
|
if (rg_sql_struct_update_needed($db) !== 0) { |
|
194 |
|
rg_log('Update needed. Exit.'); |
|
195 |
|
exit(0); |
|
196 |
|
} |
|
197 |
|
|
|
198 |
|
if (rg_fixes_needed($db) !== 0) { |
|
199 |
|
rg_log('Fixes needed. Exit.'); |
|
200 |
|
exit(0); |
|
201 |
|
} |
|
202 |
|
|
|
203 |
|
// Prepare socket |
|
204 |
|
$socket = @socket_create(AF_INET, SOCK_STREAM, SOL_TCP); |
|
205 |
|
if ($socket === FALSE) { |
|
206 |
|
rg_internal_error('Cannot create socket!'); |
|
207 |
|
exit(1); |
|
208 |
|
} |
|
209 |
|
|
|
210 |
|
socket_set_option($socket, SOL_SOCKET, SO_REUSEADDR, 1); |
|
211 |
|
|
|
212 |
|
$r = @socket_bind($socket, $rg_builder_bind, $rg_builder_port); |
|
213 |
|
if ($r === FALSE) { |
|
214 |
|
rg_internal_error("Cannot bind socket!"); |
|
215 |
|
exit(1); |
|
216 |
|
} |
|
217 |
|
|
|
218 |
|
$r = @socket_listen($socket, 128); |
|
219 |
|
if ($r === FALSE) { |
|
220 |
|
rg_internal_error("Cannot set queue length on socket!"); |
|
221 |
|
exit(1); |
|
222 |
|
} |
|
223 |
|
|
|
224 |
|
rg_conn_new('master', $socket); |
|
225 |
|
$rg_conns['master']['exit_on_close'] = 1; |
|
226 |
|
$rg_conns['master']['func_new'] = 'xnew'; |
|
227 |
|
$rg_conns['master']['func_new_arg'] = $db; |
|
228 |
|
|
|
229 |
|
$jobs = array(); |
|
230 |
|
|
|
231 |
|
$original_mtime = @filemtime(__FILE__); |
|
232 |
|
do { |
|
233 |
|
rg_log_buffer_clear(); |
|
234 |
|
|
|
235 |
|
// Check our mtime so we can upgrade the software and this script |
|
236 |
|
// will restart. |
|
237 |
|
clearstatcache(); |
|
238 |
|
$mtime = @filemtime(__FILE__); |
|
239 |
|
if ($mtime != $original_mtime) { |
|
240 |
|
rg_log("mtime=$mtime, original_mtime=$original_mtime"); |
|
241 |
|
rg_log('File changed. Exiting...'); |
|
242 |
|
break; |
|
243 |
|
} |
|
244 |
|
|
|
245 |
|
$r = rg_builder_load_jobs($db); |
|
246 |
|
if ($r['ok'] != 1) { |
|
247 |
|
rg_log('Cannot load jobs from database!'); |
|
248 |
|
sleep(30); |
|
249 |
|
continue; |
|
250 |
|
} |
|
251 |
|
|
|
252 |
|
foreach ($r['list'] as $jid => $job) { |
|
253 |
|
if (!isset($jobs[$jid])) { |
|
254 |
|
$job['worker'] = ''; |
|
255 |
|
$job['url'] = 'git://rg.embedromix.ro/user/catab/delme10'; // TODO |
|
256 |
|
$jobs[$jid] = $job; |
|
257 |
|
} |
|
258 |
|
|
|
259 |
|
$r = rg_process_queue($db, $jobs[$jid]); |
|
260 |
|
if ($r === FALSE) |
|
261 |
|
break; |
|
262 |
|
} |
|
263 |
|
if ($r === FALSE) |
|
264 |
|
break; |
|
265 |
|
|
|
266 |
|
rg_log("Waiting for connections..."); |
|
267 |
|
rg_conn_wait(10); |
|
268 |
|
} while (1); |
|
269 |
|
|
|
270 |
|
@socket_close($socket); |
|
271 |
|
|
|
272 |
|
rg_log("Exiting..."); |
|
273 |
|
|
|
274 |
|
rg_prof_end("MAIN"); |
|
275 |
|
rg_prof_log(); |
|
276 |
|
?> |
File scripts/worker.php added (mode: 100644) (index 0000000..16328b0) |
|
1 |
|
<?php |
|
2 |
|
// Client for continuous integration and deployment |
|
3 |
|
error_reporting(E_ALL); |
|
4 |
|
ini_set('track_errors', 'On'); |
|
5 |
|
set_time_limit(0); |
|
6 |
|
|
|
7 |
|
define('RG_JOB_INIT', 1); |
|
8 |
|
define('RG_JOB_HELPER_STARTED', 2); |
|
9 |
|
define('RG_JOB_STARTED', 3); |
|
10 |
|
define('RG_JOB_DONE', 10); |
|
11 |
|
|
|
12 |
|
$_s = microtime(TRUE); |
|
13 |
|
|
|
14 |
|
$INC = dirname(__FILE__) . "/../inc"; |
|
15 |
|
require_once($INC . "/init.inc.php"); |
|
16 |
|
require_once($INC . "/log.inc.php"); |
|
17 |
|
require_once($INC . "/prof.inc.php"); |
|
18 |
|
require_once($INC . "/builder.inc.php"); |
|
19 |
|
require_once($INC . "/conn.inc.php"); |
|
20 |
|
|
|
21 |
|
rg_prof_start('MAIN'); |
|
22 |
|
|
|
23 |
|
rg_log_set_file($rg_log_dir . '/worker.log'); |
|
24 |
|
rg_log_set_sid("000000"); // to spread the logs |
|
25 |
|
|
|
26 |
|
|
|
27 |
|
/* |
|
28 |
|
* Load configuration file |
|
29 |
|
*/ |
|
30 |
|
function reload_config() |
|
31 |
|
{ |
|
32 |
|
global $conf; |
|
33 |
|
|
|
34 |
|
$_conf = @file('/etc/rocketgit/worker.conf'); |
|
35 |
|
$last_key = FALSE; |
|
36 |
|
if ($_conf !== FALSE) { |
|
37 |
|
$conf = array('env' => array()); |
|
38 |
|
foreach ($_conf as $line) { |
|
39 |
|
if (empty(trim($line))) |
|
40 |
|
continue; |
|
41 |
|
|
|
42 |
|
if (strncmp($line, '#', 1) == 0) |
|
43 |
|
continue; |
|
44 |
|
|
|
45 |
|
$t = explode('=', $line, 2); |
|
46 |
|
if (count($t) != 2) { |
|
47 |
|
rg_log('Invalid line [' . $line . ']!'); |
|
48 |
|
continue; |
|
49 |
|
} |
|
50 |
|
|
|
51 |
|
$var = trim($t[0]); |
|
52 |
|
$value = trim($t[1]); |
|
53 |
|
|
|
54 |
|
if (strcmp($var, 'env') == 0) { |
|
55 |
|
$conf['env'][$value] = array('paras' => ''); |
|
56 |
|
$last_parent = &$conf['env'][$value]; |
|
57 |
|
} else if ((strncmp($line, " ", 1) == 0) |
|
58 |
|
|| (strncmp($line, "\t", 1) == 0)) { |
|
59 |
|
if ($last_parent === FALSE) { |
|
60 |
|
rg_log('Invalid line [' . $line . ']!'); |
|
61 |
|
continue; |
|
62 |
|
} |
|
63 |
|
|
|
64 |
|
$t = explode('=', $line, 2); |
|
65 |
|
if (count($t) != 2) { |
|
66 |
|
rg_log('Invalid line [' . $line . ']!'); |
|
67 |
|
continue; |
|
68 |
|
} |
|
69 |
|
|
|
70 |
|
$var = trim($t[0]); |
|
71 |
|
$value = trim($t[1]); |
|
72 |
|
$last_parent[$var] = $value; |
|
73 |
|
} else { |
|
74 |
|
$conf[$var] = $value; |
|
75 |
|
$last_parent = FALSE; |
|
76 |
|
} |
|
77 |
|
} |
|
78 |
|
} else { |
|
79 |
|
$conf = array( |
|
80 |
|
'workers' => '1', |
|
81 |
|
'cost' => 1, |
|
82 |
|
'env' => array() |
|
83 |
|
); |
|
84 |
|
} |
|
85 |
|
rg_log_ml('conf: ' . print_r($conf, TRUE)); |
|
86 |
|
} |
|
87 |
|
|
|
88 |
|
/* |
|
89 |
|
* Starts an worker |
|
90 |
|
*/ |
|
91 |
|
function start_worker($job) |
|
92 |
|
{ |
|
93 |
|
global $conf; |
|
94 |
|
global $php_errormsg; |
|
95 |
|
|
|
96 |
|
$env = $conf['env'][$job['env']]; |
|
97 |
|
//rg_log_ml('DEBUG: env: ' . print_r($env, TRUE)); |
|
98 |
|
|
|
99 |
|
$jid = $job['id']; |
|
100 |
|
$emain = escapeshellarg($job['main']); |
|
101 |
|
|
|
102 |
|
$name = 'rg-worker-' . $job['id']; |
|
103 |
|
$ename = escapeshellarg($name); |
|
104 |
|
$master = escapeshellarg($env['image']); |
|
105 |
|
$img = escapeshellarg($job['main'] . '/image.qcow2'); |
|
106 |
|
$img2 = escapeshellarg($job['main'] . '/image2.raw'); |
|
107 |
|
|
|
108 |
|
$do_umount = FALSE; |
|
109 |
|
$err = TRUE; |
|
110 |
|
while (1) { |
|
111 |
|
rg_exec('virsh destroy ' . $ename); |
|
112 |
|
rg_exec('virsh undefine ' . $ename); |
|
113 |
|
|
|
114 |
|
$r = rg_del_tree($job['main']); |
|
115 |
|
if ($r === FALSE) { |
|
116 |
|
rg_log('Cannot delete main dir (' . $job['main'] . ')!'); |
|
117 |
|
break; |
|
118 |
|
} |
|
119 |
|
|
|
120 |
|
$r = @mkdir($job['main'], 0755); |
|
121 |
|
if ($r === FALSE) { |
|
122 |
|
rg_log('Cannot create main dir (' . $job['main'] . '):' |
|
123 |
|
. ' ' . $php_errormsg . '!'); |
|
124 |
|
break; |
|
125 |
|
} |
|
126 |
|
|
|
127 |
|
// Save & confirm the receiving (TODO: fsync) |
|
128 |
|
// TODO: This will be used to clean up on a restart |
|
129 |
|
$r = @file_put_contents($job['main'] . '/job.ser', |
|
130 |
|
serialize($job)); |
|
131 |
|
if ($r === FALSE) { |
|
132 |
|
$reason = 'cannot store job'; |
|
133 |
|
break; |
|
134 |
|
} |
|
135 |
|
|
|
136 |
|
$r = rg_exec('qemu-img create -b ' . $master |
|
137 |
|
. ' -f qcow2 ' . $img); |
|
138 |
|
if ($r['ok'] !== 1) { |
|
139 |
|
$reason = 'cannot create image: ' . $r['errmsg']; |
|
140 |
|
break; |
|
141 |
|
} |
|
142 |
|
|
|
143 |
|
$r = rg_exec('qemu-img create -f raw ' . $img2 . ' 1G'); |
|
144 |
|
if ($r['ok'] !== 1) { |
|
145 |
|
$reason = 'cannot create image2: ' . $r['errmsg']; |
|
146 |
|
break; |
|
147 |
|
} |
|
148 |
|
|
|
149 |
|
$r = rg_exec('mkfs.ext4 -L RG ' . $img2); |
|
150 |
|
if ($r['ok'] !== 1) { |
|
151 |
|
$reason = 'cannot create fs: ' . $r['errmsg']; |
|
152 |
|
break; |
|
153 |
|
} |
|
154 |
|
|
|
155 |
|
$r = @mkdir($job['main'] . '/root', 0755); |
|
156 |
|
if ($r === FALSE) { |
|
157 |
|
$reason = 'Cannot create root dir: ' . $php_errormsg . '!'; |
|
158 |
|
break; |
|
159 |
|
} |
|
160 |
|
|
|
161 |
|
$r = rg_exec('mount ' . $img2 . ' ' . $emain . '/root'); |
|
162 |
|
if ($r['ok'] !== 1) { |
|
163 |
|
$reason = 'cannot mount fs: ' . $r['errmsg']; |
|
164 |
|
break; |
|
165 |
|
} |
|
166 |
|
$do_umount = TRUE; |
|
167 |
|
|
|
168 |
|
// Clone repo |
|
169 |
|
$cmd = 'git clone --depth 1' |
|
170 |
|
. ' ' . escapeshellarg($job['url']) |
|
171 |
|
. ' ' . $emain . '/root/git'; |
|
172 |
|
$r = rg_exec($cmd); |
|
173 |
|
if ($r['ok'] !== 1) { |
|
174 |
|
$reason = 'cannot clone: ' . $r['errmsg']; |
|
175 |
|
break; |
|
176 |
|
} |
|
177 |
|
|
|
178 |
|
// Build command list |
|
179 |
|
$s = 'export RG_LABELS=/mnt/status/RG_LABELS' . "\n\n"; |
|
180 |
|
$s .= 'cd /mnt/git' . "\n\n"; |
|
181 |
|
$s .= 'git checkout ' . escapeshellarg($job['head']) . "\n"; |
|
182 |
|
foreach ($job['cmds'] as $name => $i) { |
|
183 |
|
$prefix = '/mnt/status/' |
|
184 |
|
. escapeshellarg($name); |
|
185 |
|
|
|
186 |
|
if (empty($i['label_ok'])) |
|
187 |
|
$lok = ''; |
|
188 |
|
else |
|
189 |
|
$lok = ' echo ' |
|
190 |
|
. escapeshellarg($i['label_ok']) |
|
191 |
|
. ' >>/mnt/status/RG_LABELS' . "\n"; |
|
192 |
|
|
|
193 |
|
if (empty($i['label_nok'])) |
|
194 |
|
$lnok = ''; |
|
195 |
|
else |
|
196 |
|
$lnok = ' echo ' |
|
197 |
|
. escapeshellarg($i['label_nok']) |
|
198 |
|
. ' >>/mnt/status/RG_LABELS' . "\n"; |
|
199 |
|
|
|
200 |
|
$s .= 'date +%s > ' . $prefix . '.start' . "\n" |
|
201 |
|
. '(' . $i['cmd'] . ') 1>' . $prefix . '.log 2>&1' . "\n" |
|
202 |
|
. 'E=${?}' . "\n" |
|
203 |
|
. 'date +%s > ' . $prefix . '.done' . "\n" |
|
204 |
|
. 'if [ "${E}" != "0" ]; then' . "\n" |
|
205 |
|
. ' echo ${E} > ' . $prefix . ".status\n" |
|
206 |
|
. $lnok |
|
207 |
|
. ($i['abort'] ? ' exit 0' . "\n" : '') |
|
208 |
|
. 'else' . "\n" |
|
209 |
|
. ' echo 0 > ' . $prefix . ".status\n" |
|
210 |
|
. $lok |
|
211 |
|
. 'fi' . "\n\n"; |
|
212 |
|
} |
|
213 |
|
$r = @file_put_contents($job['main'] . '/root/build.sh', $s); |
|
214 |
|
if ($r === FALSE) { |
|
215 |
|
$reason = 'cannot store build commands!'; |
|
216 |
|
break; |
|
217 |
|
} |
|
218 |
|
$r = @chmod($job['main'] . '/root/build.sh', 0755); |
|
219 |
|
if ($r === FALSE) { |
|
220 |
|
$reason = 'cannot to chmod on build.sh!'; |
|
221 |
|
break; |
|
222 |
|
} |
|
223 |
|
|
|
224 |
|
// Store commands |
|
225 |
|
$r = @file_put_contents($job['main'] . '/root/rg.sh', |
|
226 |
|
'mkdir /mnt/status' . "\n" |
|
227 |
|
. 'chown -R build:build /mnt/git /mnt/status' . "\n" |
|
228 |
|
. 'date +%s > /mnt/T_START' . "\n" |
|
229 |
|
. 'su - build -c /mnt/build.sh' . "\n" |
|
230 |
|
. 'date +%s > /mnt/T_DONE' . "\n" |
|
231 |
|
. 'sync' . "\n" |
|
232 |
|
. 'shutdown -h now' |
|
233 |
|
); |
|
234 |
|
if ($r === FALSE) { |
|
235 |
|
$reason = 'cannot store commands!'; |
|
236 |
|
break; |
|
237 |
|
} |
|
238 |
|
$r = @chmod($job['main'] . '/root/rg.sh', 0755); |
|
239 |
|
if ($r === FALSE) { |
|
240 |
|
$reason = 'cannot to chmod on rg.sh!'; |
|
241 |
|
break; |
|
242 |
|
} |
|
243 |
|
|
|
244 |
|
$r = rg_exec('umount ' . $emain . '/root'); |
|
245 |
|
if ($r['ok'] !== 1) { |
|
246 |
|
$reason = 'cannot umount fs: ' . $r['errmsg']; |
|
247 |
|
break; |
|
248 |
|
} |
|
249 |
|
$do_umount = FALSE; |
|
250 |
|
|
|
251 |
|
// . ' --console pty,target_type=virtio' |
|
252 |
|
// . ' --noautoconsole' |
|
253 |
|
// . ' --security type=dynamic,relabel=yes' |
|
254 |
|
// . ' --filesystem source=' . $emain . '/root') |
|
255 |
|
// . ',target=rg,mode=mapped' // passthrough |
|
256 |
|
// . ' --security type=static,label=' . $context . ',relabel=yes' |
|
257 |
|
$r = rg_exec('virt-install' |
|
258 |
|
. ' --name ' . $ename |
|
259 |
|
. ' --arch ' . escapeshellarg($env['arch']) |
|
260 |
|
. ' --memory 256' |
|
261 |
|
. ' --vcpus 1' |
|
262 |
|
. ' --graphics none' |
|
263 |
|
. ' --network none' |
|
264 |
|
. ' --security type=dynamic' |
|
265 |
|
. ' --os-variant fedora22' |
|
266 |
|
. ' --import' |
|
267 |
|
. ' --disk path=' . $img . ',discard=unmap' |
|
268 |
|
. ' --disk path=' . $img2 . ',discard=unmap' |
|
269 |
|
. ' --rng /dev/random' |
|
270 |
|
. ' ' . $env['paras']); |
|
271 |
|
if ($r['ok'] !== 1) { |
|
272 |
|
$reason = 'cannot define and start virtual machine: ' . $r['errmsg']; |
|
273 |
|
break; |
|
274 |
|
} |
|
275 |
|
|
|
276 |
|
$err = FALSE; |
|
277 |
|
break; |
|
278 |
|
} |
|
279 |
|
if ($do_umount) |
|
280 |
|
rg_exec('umount ' . $emain . '/root'); |
|
281 |
|
|
|
282 |
|
if ($err) |
|
283 |
|
@file_put_contents($job['main'] . '/error', $reason); |
|
284 |
|
} |
|
285 |
|
|
|
286 |
|
/* |
|
287 |
|
* Handle commands |
|
288 |
|
*/ |
|
289 |
|
function xhandle($key, $cmd0) |
|
290 |
|
{ |
|
291 |
|
global $jobs; |
|
292 |
|
global $conf; |
|
293 |
|
global $pid_to_jid; |
|
294 |
|
|
|
295 |
|
$cmd = substr($cmd0, 0, 4); |
|
296 |
|
$data = stripcslashes(trim(substr($cmd0, 4))); |
|
297 |
|
$job = @unserialize($data); |
|
298 |
|
if ($job === FALSE) { |
|
299 |
|
rg_log('Cannot unserialize [' . $data . ']'); |
|
300 |
|
rg_conn_destroy($key); |
|
301 |
|
return; |
|
302 |
|
} |
|
303 |
|
|
|
304 |
|
$jid = $job['id']; |
|
305 |
|
|
|
306 |
|
if (strcmp($cmd, 'BLD ') == 0) { |
|
307 |
|
if (isset($jobs[$jid])) { |
|
308 |
|
rg_log('Job ' . $jid . ' already in queue!'); |
|
309 |
|
return; |
|
310 |
|
} |
|
311 |
|
|
|
312 |
|
$jobs[$jid] = $job; |
|
313 |
|
$jobs[$jid]['main'] = $conf['state'] . '/j-' . $jid; |
|
314 |
|
$jobs[$jid]['state'] = RG_JOB_INIT; |
|
315 |
|
|
|
316 |
|
rg_log_ml('build job: ' . print_r($job, TRUE)); |
|
317 |
|
$err = TRUE; |
|
318 |
|
while (1) { |
|
319 |
|
// Fork |
|
320 |
|
$pid = pcntl_fork(); |
|
321 |
|
if ($pid == -1) { |
|
322 |
|
$reason = 'Cannot fork!'; |
|
323 |
|
break; |
|
324 |
|
} |
|
325 |
|
if ($pid == 0) { // child |
|
326 |
|
start_worker($jobs[$jid]); |
|
327 |
|
exit(0); |
|
328 |
|
} |
|
329 |
|
|
|
330 |
|
rg_log('Started builder with pid ' . $pid); |
|
331 |
|
$jobs[$jid]['state'] = RG_JOB_HELPER_STARTED; |
|
332 |
|
$pid_to_jid[$pid] = $jid; |
|
333 |
|
$err = FALSE; |
|
334 |
|
break; |
|
335 |
|
} |
|
336 |
|
$a = array('id' => $jid); |
|
337 |
|
if ($err) { |
|
338 |
|
$a['reason'] = $reason; |
|
339 |
|
rg_conn_enq('master', 'ABR ' . rg_conn_prepare($a) . "\n"); |
|
340 |
|
unset($pid_to_jid[$pid]); |
|
341 |
|
unset($jobs[$jid]); |
|
342 |
|
return; |
|
343 |
|
} |
|
344 |
|
rg_conn_enq('master', 'STA ' . rg_conn_prepare($a) . "\n"); |
|
345 |
|
} else if (strcmp($cmd, 'DRE ') == 0) { |
|
346 |
|
rg_log('DRE command'); |
|
347 |
|
$job = &$jobs[$jid]; |
|
348 |
|
unset($pid_to_jid[$job['pid']]); |
|
349 |
|
unset($jobs[$jid]); |
|
350 |
|
@unlink($job['main'] . '/job.ser'); |
|
351 |
|
} else { |
|
352 |
|
rg_log('Cannot handle[' . $key . ']: ' . $cmd); |
|
353 |
|
} |
|
354 |
|
} |
|
355 |
|
|
|
356 |
|
/* |
|
357 |
|
* Extracts info from the virtual disk |
|
358 |
|
*/ |
|
359 |
|
function rg_job_extract_info(&$job) |
|
360 |
|
{ |
|
361 |
|
global $conf; |
|
362 |
|
|
|
363 |
|
$jid = $job['id']; |
|
364 |
|
rg_log('DEBUG: extract_info: root=' . $job['main']); |
|
365 |
|
$emain = escapeshellarg($job['main']); |
|
366 |
|
|
|
367 |
|
while (1) { |
|
368 |
|
if (!is_dir($job['main'])) { |
|
369 |
|
$job['error'] = 'Main dir not presend;' |
|
370 |
|
. ' probably disk space problems'; |
|
371 |
|
break; |
|
372 |
|
} |
|
373 |
|
|
|
374 |
|
$r = @file_get_contents($job['main'] . '/error'); |
|
375 |
|
if ($r !== FALSE) { |
|
376 |
|
$job['error'] = $r; |
|
377 |
|
break; |
|
378 |
|
} |
|
379 |
|
|
|
380 |
|
$cmd = 'mount ' . $emain . '/image2.raw ' . $emain . '/root'; |
|
381 |
|
$r = rg_exec($cmd); |
|
382 |
|
if ($r['ok'] != 1) { |
|
383 |
|
$job['error'] = 'Could not mount image: ' . $r['data']; |
|
384 |
|
break; |
|
385 |
|
} |
|
386 |
|
|
|
387 |
|
$job['status'] = array( |
|
388 |
|
'start' => @trim(file_get_contents($job['main'] . '/root/T_START')), |
|
389 |
|
'done' => @trim(file_get_contents($job['main'] . '/root/T_DONE')), |
|
390 |
|
'labels' => @file($job['main'] . '/root/status/RG_LABELS') |
|
391 |
|
); |
|
392 |
|
foreach ($job['status']['labels'] as $index => $l) |
|
393 |
|
$job['status']['labels'][$index] = trim($l); |
|
394 |
|
|
|
395 |
|
$job['status']['cmds'] = array(); |
|
396 |
|
foreach ($job['cmds'] as $cmd => $i) { |
|
397 |
|
if (empty($i['cmd'])) |
|
398 |
|
continue; |
|
399 |
|
|
|
400 |
|
$sd = $job['main'] . '/root/status/' . $cmd; |
|
401 |
|
$job['status']['cmds'][$cmd] = array( |
|
402 |
|
'start' => @trim(file_get_contents($sd . '.start')), |
|
403 |
|
'done' => @trim(file_get_contents($sd . '.done')), |
|
404 |
|
'log' => @file_get_contents($sd . '.log', FALSE, |
|
405 |
|
NULL, -1, 2048) |
|
406 |
|
); |
|
407 |
|
} |
|
408 |
|
unset($job['cmds']); |
|
409 |
|
unset($job['url']); |
|
410 |
|
unset($job['head']); |
|
411 |
|
unset($job['env']); |
|
412 |
|
|
|
413 |
|
$cmd = 'umount ' . $emain . '/root'; |
|
414 |
|
$r = rg_exec($cmd); |
|
415 |
|
if ($r['ok'] != 1) { |
|
416 |
|
rg_log('Cannot unmount!'); |
|
417 |
|
break; |
|
418 |
|
} |
|
419 |
|
|
|
420 |
|
rg_del_tree($job['main']); |
|
421 |
|
break; |
|
422 |
|
} |
|
423 |
|
|
|
424 |
|
rg_log_ml('DEBUG: job: ' . print_r($job, TRUE)); |
|
425 |
|
return TRUE; |
|
426 |
|
} |
|
427 |
|
|
|
428 |
|
|
|
429 |
|
reload_config(); |
|
430 |
|
|
|
431 |
|
$socket = @socket_create(AF_INET, SOCK_STREAM, SOL_TCP); |
|
432 |
|
if ($socket === FALSE) { |
|
433 |
|
rg_log('Cannot create socket!'); |
|
434 |
|
exit(1); |
|
435 |
|
} |
|
436 |
|
|
|
437 |
|
$r = @socket_connect($socket, $conf['master'], $conf['port']); |
|
438 |
|
if ($r === FALSE) { |
|
439 |
|
rg_log('Cannot connect to ' . $conf['master'] . '/' |
|
440 |
|
. $conf['port'] . '!'); |
|
441 |
|
exit(1); |
|
442 |
|
} |
|
443 |
|
|
|
444 |
|
rg_conn_new('master', $socket); |
|
445 |
|
$rg_conns['master']['exit_on_close'] = 1; |
|
446 |
|
$rg_conns['master']['func_cmd'] = 'xhandle'; |
|
447 |
|
|
|
448 |
|
// announce ourselves |
|
449 |
|
$key = $conf['key']; unset($conf['key']); |
|
450 |
|
$ann = $conf; |
|
451 |
|
$ann['uname'] = php_uname('a'); |
|
452 |
|
$ann['host'] = php_uname('n'); |
|
453 |
|
$ann['arch'] = php_uname('m'); |
|
454 |
|
$ann['boot_time'] = time(); |
|
455 |
|
$ann['sign'] = hash_hmac('sha512', $ann['boot_time'], $key); |
|
456 |
|
rg_conn_enq('master', 'ANN ' . rg_conn_prepare($ann) . "\n"); |
|
457 |
|
|
|
458 |
|
$jobs = array(); |
|
459 |
|
$pid_to_jid = array(); |
|
460 |
|
while(1) { |
|
461 |
|
rg_log_buffer_clear(); |
|
462 |
|
rg_conn_wait(3); |
|
463 |
|
|
|
464 |
|
// Verify if the jobs are started |
|
465 |
|
while (1) { |
|
466 |
|
$pid = pcntl_waitpid(-1, $status, WNOHANG); |
|
467 |
|
if ($pid === 0) |
|
468 |
|
break; |
|
469 |
|
if ($pid == -1) |
|
470 |
|
break; |
|
471 |
|
|
|
472 |
|
$jid = $pid_to_jid[$pid]; |
|
473 |
|
rg_log('Pid ' . $pid . ' exited (job ' . $jid . ')!'); |
|
474 |
|
unset($pid_to_jid[$pid]); |
|
475 |
|
$jobs[$jid]['state'] = RG_JOB_STARTED; |
|
476 |
|
} |
|
477 |
|
|
|
478 |
|
// Verify if vms finished |
|
479 |
|
$vms_loaded = FALSE; |
|
480 |
|
foreach ($jobs as $jid => &$job) { |
|
481 |
|
if ($job['state'] != RG_JOB_STARTED) |
|
482 |
|
continue; |
|
483 |
|
|
|
484 |
|
if ($vms_loaded === FALSE) { |
|
485 |
|
$vms = rg_builder_vm_list(); |
|
486 |
|
if ($vms === FALSE) |
|
487 |
|
break; |
|
488 |
|
|
|
489 |
|
$vms_loaded = TRUE; |
|
490 |
|
rg_log_ml('vms: ' . print_r($vms, TRUE)); |
|
491 |
|
} |
|
492 |
|
|
|
493 |
|
$name = 'rg-worker-' . $jid; |
|
494 |
|
$k = array_search($name, $vms); |
|
495 |
|
if ($k === FALSE) { |
|
496 |
|
rg_log('VM finished'); |
|
497 |
|
$job['state'] = RG_JOB_DONE; |
|
498 |
|
|
|
499 |
|
rg_job_extract_info($job); |
|
500 |
|
|
|
501 |
|
// TODO: store in fs to be able to still inform the |
|
502 |
|
// master if we are crashing. |
|
503 |
|
$job['state'] = RG_JOB_DONE; |
|
504 |
|
@file_put_contents($job['main'] . '/job.ser', |
|
505 |
|
serialize($job)); |
|
506 |
|
|
|
507 |
|
rg_conn_enq('master', 'DON ' |
|
508 |
|
. rg_conn_prepare($job) . "\n"); |
|
509 |
|
} else { |
|
510 |
|
//rg_log('VM in progress'); |
|
511 |
|
// TODO: if too much time, abort |
|
512 |
|
} |
|
513 |
|
} |
|
514 |
|
} |
|
515 |
|
|
|
516 |
|
rg_prof_end("MAIN"); |
|
517 |
|
rg_prof_log(); |
|
518 |
|
?> |