1
00:00:06,320 --> 00:00:11,499
[Music]

2
00:00:15,679 --> 00:00:18,080
welcome back

3
00:00:17,279 --> 00:00:20,400
um

4
00:00:18,080 --> 00:00:22,320
please welcome david uh who is a

5
00:00:20,400 --> 00:00:24,720
software engineer at google uh working

6
00:00:22,320 --> 00:00:27,039
on k unit a unit testing framework for

7
00:00:24,720 --> 00:00:29,279
the linux kernel and he'll be speaking

8
00:00:27,039 --> 00:00:32,320
to us today on kernel testing with k

9
00:00:29,279 --> 00:00:32,320
unit bridging the gap

10
00:00:32,640 --> 00:00:34,800
thanks

11
00:00:33,680 --> 00:00:37,280
uh

12
00:00:34,800 --> 00:00:38,559
no good afternoon everyone uh as you

13
00:00:37,280 --> 00:00:40,960
already know

14
00:00:38,559 --> 00:00:43,120
i'm david gal and i am going to be uh

15
00:00:40,960 --> 00:00:44,640
talking about kernel testing with k unit

16
00:00:43,120 --> 00:00:45,920
bridging the gap

17
00:00:44,640 --> 00:00:46,879
um

18
00:00:45,920 --> 00:00:49,680
so

19
00:00:46,879 --> 00:00:52,000
who am i as mentioned i'm david um i've

20
00:00:49,680 --> 00:00:54,399
been using linux basically full time

21
00:00:52,000 --> 00:00:56,399
since 2006 and have been working on the

22
00:00:54,399 --> 00:00:58,719
k unit kernel testing framework at

23
00:00:56,399 --> 00:01:00,879
google for the last three years

24
00:00:58,719 --> 00:01:01,920
and in this talk we're basically going

25
00:01:00,879 --> 00:01:04,080
to cover

26
00:01:01,920 --> 00:01:05,600
you know what why and how you should

27
00:01:04,080 --> 00:01:07,600
test the kernel

28
00:01:05,600 --> 00:01:09,680
and particularly look at k unit which is

29
00:01:07,600 --> 00:01:12,960
what i work on and k self-test which is

30
00:01:09,680 --> 00:01:14,159
the other main entry kernel testing

31
00:01:12,960 --> 00:01:14,960
framework

32
00:01:14,159 --> 00:01:16,799
um

33
00:01:14,960 --> 00:01:19,040
in particular we're going to cover

34
00:01:16,799 --> 00:01:21,360
what's been changing and what's improved

35
00:01:19,040 --> 00:01:24,400
both in k unit and in how it interacts

36
00:01:21,360 --> 00:01:26,080
with case self-test and how in general

37
00:01:24,400 --> 00:01:28,479
these testing frameworks work over the

38
00:01:26,080 --> 00:01:29,840
last year and where we're looking to go

39
00:01:28,479 --> 00:01:31,920
in the future

40
00:01:29,840 --> 00:01:33,200
uh what we're not going to be talking

41
00:01:31,920 --> 00:01:35,600
about uh

42
00:01:33,200 --> 00:01:38,799
is actually a step-by-step guide and how

43
00:01:35,600 --> 00:01:41,280
to write tests with k unit um there are

44
00:01:38,799 --> 00:01:42,079
lots of other talks about that uh out

45
00:01:41,280 --> 00:01:44,000
there

46
00:01:42,079 --> 00:01:47,840
um but this should provide a good

47
00:01:44,000 --> 00:01:49,759
overview of of what k unit is um and as

48
00:01:47,840 --> 00:01:52,240
a preview we'll be bragging about how

49
00:01:49,759 --> 00:01:53,840
much better our documentation is so

50
00:01:52,240 --> 00:01:55,360
there's there's plenty you can find out

51
00:01:53,840 --> 00:01:56,399
about that

52
00:01:55,360 --> 00:01:58,960
um

53
00:01:56,399 --> 00:02:00,719
so let's get started by talking about

54
00:01:58,960 --> 00:02:02,159
testing the kernel and why you should do

55
00:02:00,719 --> 00:02:03,759
it and the reason you should test the

56
00:02:02,159 --> 00:02:05,600
kernel is quite obviously because you

57
00:02:03,759 --> 00:02:07,680
want the kernel to work

58
00:02:05,600 --> 00:02:10,239
in fact you really want the kernel to

59
00:02:07,680 --> 00:02:12,239
work because uh security and reliability

60
00:02:10,239 --> 00:02:14,959
bugs are bad

61
00:02:12,239 --> 00:02:17,360
in the kernel they're really bad

62
00:02:14,959 --> 00:02:19,200
the kernel has access to literally

63
00:02:17,360 --> 00:02:22,080
everything

64
00:02:19,200 --> 00:02:23,599
and if it crashes your entire system

65
00:02:22,080 --> 00:02:25,360
goes down so

66
00:02:23,599 --> 00:02:27,520
there's a pretty good incentive not to

67
00:02:25,360 --> 00:02:29,760
break the kernel um

68
00:02:27,520 --> 00:02:31,280
similarly there have been been around

69
00:02:29,760 --> 00:02:34,000
for a long time there have been a lot of

70
00:02:31,280 --> 00:02:36,720
bugs in it over time we don't want to

71
00:02:34,000 --> 00:02:39,280
have a regression where a bug we thought

72
00:02:36,720 --> 00:02:41,200
was fixed came back and automated

73
00:02:39,280 --> 00:02:43,200
testing as has sort of been well

74
00:02:41,200 --> 00:02:44,959
described by uh

75
00:02:43,200 --> 00:02:46,720
most people in the software engineering

76
00:02:44,959 --> 00:02:49,040
field over the last

77
00:02:46,720 --> 00:02:51,200
uh several decades is a really good way

78
00:02:49,040 --> 00:02:53,280
of doing that

79
00:02:51,200 --> 00:02:55,680
so there's a little bit of history um

80
00:02:53,280 --> 00:02:58,159
kernel testing has come quite a long way

81
00:02:55,680 --> 00:02:59,440
since linus basically said

82
00:02:58,159 --> 00:03:02,000
hey

83
00:02:59,440 --> 00:03:06,080
if i can get the kernel to build it's

84
00:03:02,000 --> 00:03:08,640
good and if it then boots it's perfect

85
00:03:06,080 --> 00:03:10,319
these days we we have a lot of tools for

86
00:03:08,640 --> 00:03:12,800
testing the kernel and these largely

87
00:03:10,319 --> 00:03:14,400
started as people working on on some

88
00:03:12,800 --> 00:03:17,120
feature

89
00:03:14,400 --> 00:03:20,800
saying hey i can write a script or i can

90
00:03:17,120 --> 00:03:22,879
write a test module once modules existed

91
00:03:20,800 --> 00:03:26,159
and just execute some code check the

92
00:03:22,879 --> 00:03:27,840
results see if it makes any sense

93
00:03:26,159 --> 00:03:30,239
the downside is

94
00:03:27,840 --> 00:03:32,319
lots of people writing their own tests

95
00:03:30,239 --> 00:03:34,319
in their own way ends up with something

96
00:03:32,319 --> 00:03:36,239
of a mess there are lots of tests that

97
00:03:34,319 --> 00:03:38,959
you know each do their own things so

98
00:03:36,239 --> 00:03:40,959
case self test and and later k to unit

99
00:03:38,959 --> 00:03:42,239
are both sort of attempts to standardize

100
00:03:40,959 --> 00:03:43,120
these tests

101
00:03:42,239 --> 00:03:44,319
um

102
00:03:43,120 --> 00:03:46,000
and they're both

103
00:03:44,319 --> 00:03:47,440
as part of being standard in tree

104
00:03:46,000 --> 00:03:49,040
they're part of the linux kernel source

105
00:03:47,440 --> 00:03:50,959
tree you go to kernel.org you download

106
00:03:49,040 --> 00:03:53,280
the kernel source both k unit and k

107
00:03:50,959 --> 00:03:56,000
self-test and all of the k unit and k

108
00:03:53,280 --> 00:03:58,080
self-test tests are there in the table

109
00:03:56,000 --> 00:03:59,519
you downloaded um

110
00:03:58,080 --> 00:04:00,959
as part of being said they're also

111
00:03:59,519 --> 00:04:04,080
automatically being run on a few

112
00:04:00,959 --> 00:04:06,159
different ci systems um and

113
00:04:04,080 --> 00:04:08,159
generally the community is uh running

114
00:04:06,159 --> 00:04:10,560
these as these are the tests that you

115
00:04:08,159 --> 00:04:12,799
run as a kernel developer

116
00:04:10,560 --> 00:04:16,639
um there are other kernel testing

117
00:04:12,799 --> 00:04:18,799
systems that are not sort of internal um

118
00:04:16,639 --> 00:04:20,639
these are also very useful but we're

119
00:04:18,799 --> 00:04:22,880
going to focus on the internal ones

120
00:04:20,639 --> 00:04:23,600
today

121
00:04:22,880 --> 00:04:26,800
so

122
00:04:23,600 --> 00:04:29,040
i've mentioned case self-test and k-unit

123
00:04:26,800 --> 00:04:31,840
and sort of the big question that no

124
00:04:29,040 --> 00:04:33,520
one's had a a really easy answer to is

125
00:04:31,840 --> 00:04:34,720
when should i use one and when should i

126
00:04:33,520 --> 00:04:35,759
use the other

127
00:04:34,720 --> 00:04:36,800
um

128
00:04:35,759 --> 00:04:38,639
so

129
00:04:36,800 --> 00:04:41,600
basically that's the question i want to

130
00:04:38,639 --> 00:04:45,040
start off by trying to answer

131
00:04:41,600 --> 00:04:47,440
case self test is basically a bunch of

132
00:04:45,040 --> 00:04:49,680
scripts that run in user space each test

133
00:04:47,440 --> 00:04:50,479
is a script

134
00:04:49,680 --> 00:04:53,919
that

135
00:04:50,479 --> 00:04:55,199
exercises some kernel functionality uh

136
00:04:53,919 --> 00:04:57,520
just by

137
00:04:55,199 --> 00:04:58,639
executing whatever's accessible so it

138
00:04:57,520 --> 00:05:02,720
might be

139
00:04:58,639 --> 00:05:05,120
um you know a syscall um it might be

140
00:05:02,720 --> 00:05:07,759
reading something from csfs it might be

141
00:05:05,120 --> 00:05:10,240
you know say you wanted to test ext4 as

142
00:05:07,759 --> 00:05:12,160
a file system uh with k self-test the

143
00:05:10,240 --> 00:05:14,080
way you would do that would be have a

144
00:05:12,160 --> 00:05:16,080
test that mounts a file system image

145
00:05:14,080 --> 00:05:17,840
that you've got and does some checks on

146
00:05:16,080 --> 00:05:19,520
that from user space

147
00:05:17,840 --> 00:05:21,840
you're getting sort of a whole view of

148
00:05:19,520 --> 00:05:24,639
the system but not much view of exactly

149
00:05:21,840 --> 00:05:26,560
internally how it works

150
00:05:24,639 --> 00:05:28,960
k self test is really convenient but you

151
00:05:26,560 --> 00:05:30,960
can't access anything that is

152
00:05:28,960 --> 00:05:33,039
inside the kernel like you can't just

153
00:05:30,960 --> 00:05:34,240
say i want to call this kernel function

154
00:05:33,039 --> 00:05:36,240
directly

155
00:05:34,240 --> 00:05:38,639
or i want to read the contents of this

156
00:05:36,240 --> 00:05:40,880
variable in my tests um if you want to

157
00:05:38,639 --> 00:05:42,560
do that you need some kernel code and

158
00:05:40,880 --> 00:05:44,960
there are some case self-test tests that

159
00:05:42,560 --> 00:05:47,520
have more kernel modules that they build

160
00:05:44,960 --> 00:05:49,199
and run to expose these things but the

161
00:05:47,520 --> 00:05:51,039
better way to handle that sort of thing

162
00:05:49,199 --> 00:05:53,280
where you want to test individual kernel

163
00:05:51,039 --> 00:05:55,360
functions or individual kernel variables

164
00:05:53,280 --> 00:05:57,280
have the correct value

165
00:05:55,360 --> 00:06:00,080
is to use k unit

166
00:05:57,280 --> 00:06:02,240
and k unit is basically designed as a

167
00:06:00,080 --> 00:06:04,560
unit testing framework so you're going

168
00:06:02,240 --> 00:06:07,120
to be testing like individual c

169
00:06:04,560 --> 00:06:08,160
functions uh within the kernel

170
00:06:07,120 --> 00:06:10,479
um

171
00:06:08,160 --> 00:06:12,720
and it has the advantage that all of the

172
00:06:10,479 --> 00:06:15,600
test code is part of the kernel it's

173
00:06:12,720 --> 00:06:17,919
built into your vm linux.so

174
00:06:15,600 --> 00:06:21,120
your vm linux file or it can be loaded

175
00:06:17,919 --> 00:06:23,120
as a kernel module itself directly

176
00:06:21,120 --> 00:06:24,960
you can you want to call a kernel

177
00:06:23,120 --> 00:06:25,759
function you can just call it

178
00:06:24,960 --> 00:06:27,360
um

179
00:06:25,759 --> 00:06:29,680
this does make it a bit trickier to

180
00:06:27,360 --> 00:06:32,319
write those big integration tests that

181
00:06:29,680 --> 00:06:34,639
that cover the whole journey from you

182
00:06:32,319 --> 00:06:36,319
run mount to the file has the contents

183
00:06:34,639 --> 00:06:38,000
you want but it means if you want to

184
00:06:36,319 --> 00:06:40,319
test something like you know the code

185
00:06:38,000 --> 00:06:42,720
that passes timestamps in the directory

186
00:06:40,319 --> 00:06:44,880
entry or something you can have a test

187
00:06:42,720 --> 00:06:47,120
that tests just that function and passes

188
00:06:44,880 --> 00:06:49,280
all the possible inputs that you know

189
00:06:47,120 --> 00:06:53,199
would be awkward to have in like a test

190
00:06:49,280 --> 00:06:54,479
file system and verifies them one by one

191
00:06:53,199 --> 00:06:57,440
this means that things have to be

192
00:06:54,479 --> 00:06:59,360
written exactly in c like other kernel

193
00:06:57,440 --> 00:07:01,039
things or i guess rust in the future or

194
00:06:59,360 --> 00:07:03,840
assembly if you really really hate

195
00:07:01,039 --> 00:07:05,840
yourself but um

196
00:07:03,840 --> 00:07:06,800
it also means that

197
00:07:05,840 --> 00:07:08,800
you know

198
00:07:06,800 --> 00:07:11,520
if you can use the kernel it works there

199
00:07:08,800 --> 00:07:13,919
are no extra dependencies

200
00:07:11,520 --> 00:07:15,599
now so that's really when you should use

201
00:07:13,919 --> 00:07:17,039
case self-testing k unit are you trying

202
00:07:15,599 --> 00:07:18,880
to check

203
00:07:17,039 --> 00:07:20,639
sort of the external behavior of the

204
00:07:18,880 --> 00:07:23,280
system or are you trying to attack

205
00:07:20,639 --> 00:07:25,440
individual functions

206
00:07:23,280 --> 00:07:27,840
there are some other testing tools um

207
00:07:25,440 --> 00:07:29,599
that the kernel provides that

208
00:07:27,840 --> 00:07:31,919
are useful to know about but don't

209
00:07:29,599 --> 00:07:34,000
actually do the same sort of thing so

210
00:07:31,919 --> 00:07:35,840
either a bunch of dynamic analysis tools

211
00:07:34,000 --> 00:07:39,360
address sanitizers concurrency

212
00:07:35,840 --> 00:07:41,840
sanitizers memory leak checkers etc um

213
00:07:39,360 --> 00:07:45,280
but these are to some extent orthogonal

214
00:07:41,840 --> 00:07:47,199
to k unit and k self-test instead of

215
00:07:45,280 --> 00:07:49,280
testing that a particular bit of code

216
00:07:47,199 --> 00:07:51,599
works and gives the correct answer

217
00:07:49,280 --> 00:07:52,960
they're just looking for common classes

218
00:07:51,599 --> 00:07:54,960
of failure

219
00:07:52,960 --> 00:07:57,120
like you know memory corruption

220
00:07:54,960 --> 00:07:59,599
accessing things you shouldn't um

221
00:07:57,120 --> 00:08:01,759
deadlocks etc um

222
00:07:59,599 --> 00:08:03,759
this means that so the way i like to

223
00:08:01,759 --> 00:08:06,240
think about it is a test like k unit and

224
00:08:03,759 --> 00:08:07,680
k self test is something that can pass

225
00:08:06,240 --> 00:08:10,560
as well as fail

226
00:08:07,680 --> 00:08:11,680
whereas these um sanitizers

227
00:08:10,560 --> 00:08:13,199
you know they're not testing a

228
00:08:11,680 --> 00:08:14,720
particular bit of code they'll just run

229
00:08:13,199 --> 00:08:16,960
the entire length of time that the

230
00:08:14,720 --> 00:08:18,479
kernel is running and they might fail if

231
00:08:16,960 --> 00:08:19,840
something goes wrong but you can never

232
00:08:18,479 --> 00:08:22,560
use them to say

233
00:08:19,840 --> 00:08:24,160
this is passed i know it's all good

234
00:08:22,560 --> 00:08:25,680
similarly there are code coverage tools

235
00:08:24,160 --> 00:08:27,680
i suspect everyone's got a pretty good

236
00:08:25,680 --> 00:08:30,160
idea what code coverage is

237
00:08:27,680 --> 00:08:31,280
uh and as with many things there are two

238
00:08:30,160 --> 00:08:33,919
different ones you can use with the

239
00:08:31,280 --> 00:08:35,279
kernel um

240
00:08:33,919 --> 00:08:38,839
you know again these are a useful tool

241
00:08:35,279 --> 00:08:41,839
to have alongside k unit okay self-test

242
00:08:38,839 --> 00:08:44,640
tests that's an awful lot of different

243
00:08:41,839 --> 00:08:46,959
information about sort of what um

244
00:08:44,640 --> 00:08:48,959
tools there exists so

245
00:08:46,959 --> 00:08:51,600
it's understandably confusing and one of

246
00:08:48,959 --> 00:08:53,200
the the things that we've done on the k

247
00:08:51,600 --> 00:08:54,640
unit side in the last year has actually

248
00:08:53,200 --> 00:08:56,480
written all of this down and put it in

249
00:08:54,640 --> 00:08:58,399
the kernel documentation

250
00:08:56,480 --> 00:09:01,360
so the first sort of great achievement

251
00:08:58,399 --> 00:09:03,120
of the last year is actually this is

252
00:09:01,360 --> 00:09:04,800
written there's now the kernel testing

253
00:09:03,120 --> 00:09:06,880
guide in the kernel documentation with a

254
00:09:04,800 --> 00:09:08,560
happy link there that goes through what

255
00:09:06,880 --> 00:09:10,720
all of these things are and when you

256
00:09:08,560 --> 00:09:12,480
should use each in a little bit more

257
00:09:10,720 --> 00:09:13,519
detail than we can squeeze into half an

258
00:09:12,480 --> 00:09:16,000
hour here

259
00:09:13,519 --> 00:09:18,240
um and has all of the appropriate links

260
00:09:16,000 --> 00:09:20,320
uh spread through so

261
00:09:18,240 --> 00:09:22,720
uh if there's one thing you should learn

262
00:09:20,320 --> 00:09:24,240
from this talk it's really

263
00:09:22,720 --> 00:09:26,160
if you want to know how to test the

264
00:09:24,240 --> 00:09:28,880
kernel there's now a document telling

265
00:09:26,160 --> 00:09:28,880
you how to do it

266
00:09:29,519 --> 00:09:33,440
so we've got these two

267
00:09:31,440 --> 00:09:35,279
kernel testing systems in case self test

268
00:09:33,440 --> 00:09:36,399
and k unit um

269
00:09:35,279 --> 00:09:37,920
and

270
00:09:36,399 --> 00:09:40,000
trying to get people to use the right

271
00:09:37,920 --> 00:09:41,680
one for the right purpose and there have

272
00:09:40,000 --> 00:09:43,200
been some challenges over the last year

273
00:09:41,680 --> 00:09:45,680
and doing that and

274
00:09:43,200 --> 00:09:47,920
the first thing really is around

275
00:09:45,680 --> 00:09:49,920
integration um

276
00:09:47,920 --> 00:09:52,240
cayenne and self-test are in a lot of

277
00:09:49,920 --> 00:09:53,600
ways very similar and serve very similar

278
00:09:52,240 --> 00:09:54,399
purposes

279
00:09:53,600 --> 00:09:56,560
and

280
00:09:54,399 --> 00:09:58,480
the same people probably should be using

281
00:09:56,560 --> 00:10:00,000
both uh we don't want to be in the

282
00:09:58,480 --> 00:10:01,600
situation where someone says well i like

283
00:10:00,000 --> 00:10:04,000
k self-test better i'm going to use it

284
00:10:01,600 --> 00:10:06,800
for everything instead of k unit or vice

285
00:10:04,000 --> 00:10:08,000
versa um because you know they're

286
00:10:06,800 --> 00:10:10,079
different enough they don't want to need

287
00:10:08,000 --> 00:10:12,320
to learn how to use both so one of our

288
00:10:10,079 --> 00:10:14,720
goals has been to basically bring these

289
00:10:12,320 --> 00:10:17,200
closer together in the areas where it

290
00:10:14,720 --> 00:10:18,480
doesn't really matter

291
00:10:17,200 --> 00:10:20,240
you know where the differences don't

292
00:10:18,480 --> 00:10:22,320
count so

293
00:10:20,240 --> 00:10:23,839
one of these is you want to be able to

294
00:10:22,320 --> 00:10:26,240
put a test from one to the other and get

295
00:10:23,839 --> 00:10:29,120
results in a similar format um you want

296
00:10:26,240 --> 00:10:31,360
the same ci systems and same pieces of

297
00:10:29,120 --> 00:10:33,519
tooling to be able to handle uh results

298
00:10:31,360 --> 00:10:34,959
from case self-test and k-unit the same

299
00:10:33,519 --> 00:10:36,160
way

300
00:10:34,959 --> 00:10:37,760
the other thing is there were some

301
00:10:36,160 --> 00:10:40,000
feature gaps that k unit and in

302
00:10:37,760 --> 00:10:42,320
particular k-units tooling has had

303
00:10:40,000 --> 00:10:44,320
compared to k-self-test

304
00:10:42,320 --> 00:10:47,040
and the main one here is that k-units

305
00:10:44,320 --> 00:10:50,000
tooling was entirely based around

306
00:10:47,040 --> 00:10:53,440
this thing called user mode linux or uml

307
00:10:50,000 --> 00:10:55,680
and uml is basically a custom

308
00:10:53,440 --> 00:10:58,399
kernel architecture

309
00:10:55,680 --> 00:11:00,000
which compiles the entire kernel as a

310
00:10:58,399 --> 00:11:03,279
user mode

311
00:11:00,000 --> 00:11:05,839
normal x86 linux executable which is

312
00:11:03,279 --> 00:11:08,399
really neat and works really fast so you

313
00:11:05,839 --> 00:11:10,160
can run your tests with k unit extremely

314
00:11:08,399 --> 00:11:12,240
quickly

315
00:11:10,160 --> 00:11:13,519
but totally lets you down if you want to

316
00:11:12,240 --> 00:11:15,120
test something that's architecture

317
00:11:13,519 --> 00:11:18,000
specific or that

318
00:11:15,120 --> 00:11:19,839
involves some sort of hardware that uml

319
00:11:18,000 --> 00:11:21,839
can't expose

320
00:11:19,839 --> 00:11:23,440
and while uml is improving a lot in what

321
00:11:21,839 --> 00:11:25,600
sorts of hardware it can expose and

322
00:11:23,440 --> 00:11:28,160
emulate

323
00:11:25,600 --> 00:11:30,880
this was really a gap between k unit

324
00:11:28,160 --> 00:11:32,160
particularly its tooling and k self-test

325
00:11:30,880 --> 00:11:34,320
so that's something else we wanted to

326
00:11:32,160 --> 00:11:35,839
address in the last year

327
00:11:34,320 --> 00:11:38,240
so let's start with the first one and

328
00:11:35,839 --> 00:11:39,600
talk about k-tap

329
00:11:38,240 --> 00:11:41,760
so

330
00:11:39,600 --> 00:11:43,920
when you have a bunch of tests you need

331
00:11:41,760 --> 00:11:46,959
to output their results sometimes

332
00:11:43,920 --> 00:11:49,760
whether they passed or failed um and

333
00:11:46,959 --> 00:11:51,760
before the days of a case self-test and

334
00:11:49,760 --> 00:11:54,160
the like this was done basically by

335
00:11:51,760 --> 00:11:56,480
people's scripts doing whatever the

336
00:11:54,160 --> 00:11:57,839
person who wrote the script thought made

337
00:11:56,480 --> 00:12:00,560
sense at the time

338
00:11:57,839 --> 00:12:02,959
so some tests would print out

339
00:12:00,560 --> 00:12:04,720
all was well some would return a

340
00:12:02,959 --> 00:12:06,480
particular error code some would print

341
00:12:04,720 --> 00:12:09,360
out nothing on success and just error

342
00:12:06,480 --> 00:12:11,839
messages on failure

343
00:12:09,360 --> 00:12:14,160
and unless you knew what that specific

344
00:12:11,839 --> 00:12:15,200
uh test script did you know it'd be

345
00:12:14,160 --> 00:12:17,519
different from everything else you'd

346
00:12:15,200 --> 00:12:20,320
have no clue um

347
00:12:17,519 --> 00:12:22,959
so it was agreed to standardize all of

348
00:12:20,320 --> 00:12:25,920
the test results with k self-test and

349
00:12:22,959 --> 00:12:26,959
other related testing tools on

350
00:12:25,920 --> 00:12:29,920
tap

351
00:12:26,959 --> 00:12:31,839
now tap is the test anything protocol

352
00:12:29,920 --> 00:12:34,800
and is this very simple

353
00:12:31,839 --> 00:12:36,560
uh output format for test results that i

354
00:12:34,800 --> 00:12:38,560
think originally came from perl or

355
00:12:36,560 --> 00:12:40,880
something

356
00:12:38,560 --> 00:12:43,839
it's very simple it's basically just a

357
00:12:40,880 --> 00:12:45,360
small header with tap version whatever

358
00:12:43,839 --> 00:12:47,279
um

359
00:12:45,360 --> 00:12:49,360
a list of how many tests and a bunch of

360
00:12:47,279 --> 00:12:50,959
lines saying okay

361
00:12:49,360 --> 00:12:54,959
test one

362
00:12:50,959 --> 00:12:58,160
test name not okay test two just okay or

363
00:12:54,959 --> 00:13:00,639
not okay for each test um and a couple

364
00:12:58,160 --> 00:13:02,560
of slightly more interesting things for

365
00:13:00,639 --> 00:13:04,959
a couple of other cases

366
00:13:02,560 --> 00:13:06,800
but in general extremely simple and

367
00:13:04,959 --> 00:13:10,320
worked pretty well

368
00:13:06,800 --> 00:13:13,760
doesn't upset anyone's human readable

369
00:13:10,320 --> 00:13:17,360
except it's a bit too simple um

370
00:13:13,760 --> 00:13:19,360
k self test and later k unit both wanted

371
00:13:17,360 --> 00:13:21,120
to do things and the big one is have

372
00:13:19,360 --> 00:13:22,959
nested tests where you can have tests

373
00:13:21,120 --> 00:13:25,279
that have sub tests

374
00:13:22,959 --> 00:13:26,720
um and so you can have a single result

375
00:13:25,279 --> 00:13:28,480
file with all of your tests that are

376
00:13:26,720 --> 00:13:30,079
nicely categorized

377
00:13:28,480 --> 00:13:33,440
um and

378
00:13:30,079 --> 00:13:34,480
tap had no support whatsoever

379
00:13:33,440 --> 00:13:36,480
for

380
00:13:34,480 --> 00:13:37,839
having nested tests

381
00:13:36,480 --> 00:13:41,279
so

382
00:13:37,839 --> 00:13:44,079
every test and every test system tried

383
00:13:41,279 --> 00:13:46,320
to hack nested tests into tap in their

384
00:13:44,079 --> 00:13:49,279
own way and we ended up basically back

385
00:13:46,320 --> 00:13:52,720
where we started

386
00:13:49,279 --> 00:13:55,360
in k units case we found a draft version

387
00:13:52,720 --> 00:13:57,279
14 of the tap spec that had nested test

388
00:13:55,360 --> 00:14:00,320
support and implemented something that

389
00:13:57,279 --> 00:14:02,079
was almost but not quite that

390
00:14:00,320 --> 00:14:04,880
and tried to convince everyone to use

391
00:14:02,079 --> 00:14:07,600
them and it turns out nobody liked tap

392
00:14:04,880 --> 00:14:10,720
14 even the people who made it um

393
00:14:07,600 --> 00:14:11,839
it was abandoned at some point um

394
00:14:10,720 --> 00:14:13,920
the actual

395
00:14:11,839 --> 00:14:14,959
spec text was under a slightly weird

396
00:14:13,920 --> 00:14:17,360
license

397
00:14:14,959 --> 00:14:19,519
uh it was not a great uh

398
00:14:17,360 --> 00:14:21,519
base to to build off

399
00:14:19,519 --> 00:14:24,079
um

400
00:14:21,519 --> 00:14:25,839
so it was decided and uh tim bird from

401
00:14:24,079 --> 00:14:27,279
sony sort of really got the ball rolling

402
00:14:25,839 --> 00:14:29,440
here that we should come up with our own

403
00:14:27,279 --> 00:14:31,519
specification based on tap

404
00:14:29,440 --> 00:14:34,800
that would have everything that the

405
00:14:31,519 --> 00:14:36,480
colonel specifically needed in it

406
00:14:34,800 --> 00:14:38,240
and there were some discussions around

407
00:14:36,480 --> 00:14:41,040
that uh

408
00:14:38,240 --> 00:14:43,120
last year and a bit the year before

409
00:14:41,040 --> 00:14:45,040
what we'd do

410
00:14:43,120 --> 00:14:48,320
and we picked it up again particularly

411
00:14:45,040 --> 00:14:50,800
one of our interns at google raymore to

412
00:14:48,320 --> 00:14:54,000
finish off and write a new full

413
00:14:50,800 --> 00:14:54,959
specification for this output format uh

414
00:14:54,000 --> 00:14:56,480
which

415
00:14:54,959 --> 00:14:58,240
rather than being something completely

416
00:14:56,480 --> 00:15:00,000
new would rather basically just

417
00:14:58,240 --> 00:15:03,040
standardize what okay self-test and

418
00:15:00,000 --> 00:15:06,720
k-unit tests are mostly already doing

419
00:15:03,040 --> 00:15:08,639
and that tools mostly can already parse

420
00:15:06,720 --> 00:15:10,240
and just shave off a few of the the

421
00:15:08,639 --> 00:15:12,480
sharp edges

422
00:15:10,240 --> 00:15:14,959
so we've got something that mostly works

423
00:15:12,480 --> 00:15:17,120
already and is a reasonably solid base

424
00:15:14,959 --> 00:15:19,360
to build off once those uh

425
00:15:17,120 --> 00:15:21,680
few things that need changing have been

426
00:15:19,360 --> 00:15:23,440
changed to match it uh it also got rid

427
00:15:21,680 --> 00:15:25,760
of some unnecessary stuff like nobody

428
00:15:23,440 --> 00:15:27,760
wanted embedded yaml or json in the

429
00:15:25,760 --> 00:15:30,160
kernel even though the tap spec

430
00:15:27,760 --> 00:15:32,880
technically supported it

431
00:15:30,160 --> 00:15:36,800
so this is what a tap

432
00:15:32,880 --> 00:15:38,720
ktab output looks like it's very basic

433
00:15:36,800 --> 00:15:40,399
i'm not going to go into in detail as

434
00:15:38,720 --> 00:15:41,680
we've all got better things to do but

435
00:15:40,399 --> 00:15:42,639
it's basically

436
00:15:41,680 --> 00:15:45,440
um

437
00:15:42,639 --> 00:15:47,680
you've got a header line a plan line and

438
00:15:45,440 --> 00:15:49,199
you can nest test results you've got an

439
00:15:47,680 --> 00:15:51,360
entire nested

440
00:15:49,199 --> 00:15:54,079
set of test results in this case there

441
00:15:51,360 --> 00:15:55,759
are 36 tests which all have interesting

442
00:15:54,079 --> 00:15:57,360
list test names

443
00:15:55,759 --> 00:15:58,240
and they all passed

444
00:15:57,360 --> 00:16:00,320
and

445
00:15:58,240 --> 00:16:03,199
the bottom one also passed

446
00:16:00,320 --> 00:16:04,320
um which is the sort of aggregate of all

447
00:16:03,199 --> 00:16:06,399
of these

448
00:16:04,320 --> 00:16:09,360
all the subtests passed therefore the

449
00:16:06,399 --> 00:16:10,399
test as a whole past a more complicated

450
00:16:09,360 --> 00:16:11,199
example

451
00:16:10,399 --> 00:16:13,360
is

452
00:16:11,199 --> 00:16:15,759
this one which has some tests which are

453
00:16:13,360 --> 00:16:17,199
skipped we'll we'll get to skip tests

454
00:16:15,759 --> 00:16:19,360
later um

455
00:16:17,199 --> 00:16:22,079
but you can you can mark tests as having

456
00:16:19,360 --> 00:16:23,920
been skipped tests that have failed uh

457
00:16:22,079 --> 00:16:25,040
comments which are called uh

458
00:16:23,920 --> 00:16:28,560
um

459
00:16:25,040 --> 00:16:30,800
you know these uh diagnostic lines um

460
00:16:28,560 --> 00:16:33,279
some statistics about how many subtests

461
00:16:30,800 --> 00:16:36,560
passed and failed uh and because

462
00:16:33,279 --> 00:16:38,639
something failed the whole thing failed

463
00:16:36,560 --> 00:16:40,399
and this is reasonably nice and human

464
00:16:38,639 --> 00:16:41,600
readable

465
00:16:40,399 --> 00:16:44,560
but

466
00:16:41,600 --> 00:16:47,920
kunit already had a bunch of tooling for

467
00:16:44,560 --> 00:16:50,720
parsing its own variant of tap

468
00:16:47,920 --> 00:16:53,839
and we want to expand that so that it

469
00:16:50,720 --> 00:16:56,160
can be used in k self-test as well uh

470
00:16:53,839 --> 00:16:57,759
and so there's this script it's part of

471
00:16:56,160 --> 00:17:00,720
uh k unit and therefore part of the

472
00:16:57,759 --> 00:17:01,759
kernel tree uh tools testing k unit k

473
00:17:00,720 --> 00:17:03,759
dot pi

474
00:17:01,759 --> 00:17:06,160
run the pars option pass that in and you

475
00:17:03,759 --> 00:17:09,120
get a nice prettily printed uh

476
00:17:06,160 --> 00:17:10,720
summary of the results

477
00:17:09,120 --> 00:17:12,720
you know it'll it'll print these

478
00:17:10,720 --> 00:17:14,000
diagnostic comments but only for tests

479
00:17:12,720 --> 00:17:15,760
that have failed it will give the

480
00:17:14,000 --> 00:17:18,559
statistics even if the test didn't print

481
00:17:15,760 --> 00:17:20,079
any statistic lines itself um

482
00:17:18,559 --> 00:17:22,559
it's pretty useful we're really glad

483
00:17:20,079 --> 00:17:25,760
that um k self-test will be able to use

484
00:17:22,559 --> 00:17:29,039
this uh in the future and again that uh

485
00:17:25,760 --> 00:17:30,640
uh ray moore who is our intern rewrote

486
00:17:29,039 --> 00:17:32,559
all of the parser for this to make it

487
00:17:30,640 --> 00:17:35,520
robust against things okay self-test did

488
00:17:32,559 --> 00:17:35,520
but k unit didn't

489
00:17:35,760 --> 00:17:40,960
the other big feature is qe mu support

490
00:17:38,880 --> 00:17:42,000
uh in k unit tool

491
00:17:40,960 --> 00:17:44,320
so

492
00:17:42,000 --> 00:17:45,840
k unit itself as i mentioned works on

493
00:17:44,320 --> 00:17:48,720
all architectures the kernel supports

494
00:17:45,840 --> 00:17:50,559
it's just more c kernel code

495
00:17:48,720 --> 00:17:52,000
but this k unit tool

496
00:17:50,559 --> 00:17:54,559
which will automatically build the

497
00:17:52,000 --> 00:17:57,679
kernel for you and run it and pass the

498
00:17:54,559 --> 00:17:59,919
output and print those nice uh summaries

499
00:17:57,679 --> 00:18:02,400
was specifically based on this user mode

500
00:17:59,919 --> 00:18:05,600
linux architecture

501
00:18:02,400 --> 00:18:07,120
so we've fixed that k unit tool now can

502
00:18:05,600 --> 00:18:09,039
cross compile

503
00:18:07,120 --> 00:18:11,919
to other architectures there's a cross

504
00:18:09,039 --> 00:18:13,200
compile flag you can just pass a uh

505
00:18:11,919 --> 00:18:15,840
gnu

506
00:18:13,200 --> 00:18:15,840
a triplet in

507
00:18:15,919 --> 00:18:20,720
and that will let you pick a compiler

508
00:18:18,240 --> 00:18:22,880
there's also an arch option which you

509
00:18:20,720 --> 00:18:24,559
pass an architecture in and there are a

510
00:18:22,880 --> 00:18:26,799
bunch of pre-written configs that will

511
00:18:24,559 --> 00:18:28,880
cross-compile the kernel for you using

512
00:18:26,799 --> 00:18:32,240
that particular architecture

513
00:18:28,880 --> 00:18:35,440
and then run it under qemu

514
00:18:32,240 --> 00:18:38,480
and extract the test results

515
00:18:35,440 --> 00:18:41,200
parse them print a nice summary so that

516
00:18:38,480 --> 00:18:43,120
means basically you can create your

517
00:18:41,200 --> 00:18:45,840
tests and then oh i want to test these

518
00:18:43,120 --> 00:18:48,640
on s390 oh i want to test these on x86

519
00:18:45,840 --> 00:18:50,720
oh i want to test these on

520
00:18:48,640 --> 00:18:52,880
you know a big indian system a powerpc

521
00:18:50,720 --> 00:18:55,200
one or something and it's it's possible

522
00:18:52,880 --> 00:18:56,799
very easily

523
00:18:55,200 --> 00:18:58,799
at the moment we support quite a few

524
00:18:56,799 --> 00:19:00,960
architectures all of the the really

525
00:18:58,799 --> 00:19:04,559
popular ones i think um

526
00:19:00,960 --> 00:19:08,559
x86 um alpha power pc

527
00:19:04,559 --> 00:19:12,080
risc-5 s390 um there are actually some

528
00:19:08,559 --> 00:19:13,600
s390 specific k unit tests um

529
00:19:12,080 --> 00:19:15,919
and it's quite easy to add new

530
00:19:13,600 --> 00:19:18,240
architectures um

531
00:19:15,919 --> 00:19:20,400
there's just a python script containing

532
00:19:18,240 --> 00:19:22,880
uh a whole bunch of config options for

533
00:19:20,400 --> 00:19:24,799
the k config options the arguments that

534
00:19:22,880 --> 00:19:28,880
need to be passed to qemu what all of

535
00:19:24,799 --> 00:19:29,840
the compiler names are by default uh etc

536
00:19:28,880 --> 00:19:31,440
um

537
00:19:29,840 --> 00:19:33,440
and this you know

538
00:19:31,440 --> 00:19:35,360
i i kind of say how useful it is to be

539
00:19:33,440 --> 00:19:37,760
able to just quickly test on another

540
00:19:35,360 --> 00:19:40,320
architecture in particular a real one

541
00:19:37,760 --> 00:19:42,880
rather than the uml um

542
00:19:40,320 --> 00:19:44,640
sort of virtualized run as a user mode

543
00:19:42,880 --> 00:19:46,799
process one

544
00:19:44,640 --> 00:19:49,840
which is fast but but does behave a

545
00:19:46,799 --> 00:19:49,840
little bit differently

546
00:19:51,120 --> 00:19:57,200
there are also a bunch of other um

547
00:19:54,240 --> 00:19:59,600
new features in k unit um

548
00:19:57,200 --> 00:20:02,720
that some are more related to tap and

549
00:19:59,600 --> 00:20:05,840
some aren't um

550
00:20:02,720 --> 00:20:08,320
so since last year's lca

551
00:20:05,840 --> 00:20:09,919
there are a bunch of of these new things

552
00:20:08,320 --> 00:20:12,720
the first one is that we've got a

553
00:20:09,919 --> 00:20:13,760
release notes page that lists them um

554
00:20:12,720 --> 00:20:15,520
so

555
00:20:13,760 --> 00:20:18,080
if you've been following k unit you

556
00:20:15,520 --> 00:20:20,159
should definitely check in on that

557
00:20:18,080 --> 00:20:21,520
to see what's been introduced in what

558
00:20:20,159 --> 00:20:22,559
new versions

559
00:20:21,520 --> 00:20:25,440
um

560
00:20:22,559 --> 00:20:27,360
it still i think says 516 has isn't out

561
00:20:25,440 --> 00:20:29,360
yet as we need to update it but it's you

562
00:20:27,360 --> 00:20:30,159
know got all of the the actual features

563
00:20:29,360 --> 00:20:31,200
there

564
00:20:30,159 --> 00:20:33,520
um

565
00:20:31,200 --> 00:20:36,720
and those include tests can now be

566
00:20:33,520 --> 00:20:40,400
skipped we talked about this last year

567
00:20:36,720 --> 00:20:42,960
it exists uh now it's actually upstream

568
00:20:40,400 --> 00:20:44,880
you can use it um the test statistics

569
00:20:42,960 --> 00:20:46,720
stuff i pointed out print statistics

570
00:20:44,880 --> 00:20:48,880
there even when you're not using k unit

571
00:20:46,720 --> 00:20:50,000
tool and there were a few people who had

572
00:20:48,880 --> 00:20:53,679
the need to print those sort of

573
00:20:50,000 --> 00:20:56,720
statistics out as a blocker for uh using

574
00:20:53,679 --> 00:20:58,080
k unit instead of whatever custom script

575
00:20:56,720 --> 00:20:59,600
they'd done before

576
00:20:58,080 --> 00:21:01,440
uh integration with the under heart iron

577
00:20:59,600 --> 00:21:03,840
behavior sanitizer

578
00:21:01,440 --> 00:21:05,520
the in documentation i i've mentioned a

579
00:21:03,840 --> 00:21:07,520
couple times has been drastically

580
00:21:05,520 --> 00:21:10,080
improved in the last year uh it's been

581
00:21:07,520 --> 00:21:12,480
totally reorganized if you go into

582
00:21:10,080 --> 00:21:14,240
documentation dev tools k unit in the

583
00:21:12,480 --> 00:21:15,200
the kernel source

584
00:21:14,240 --> 00:21:16,720
um

585
00:21:15,200 --> 00:21:18,400
there's now a better architecture

586
00:21:16,720 --> 00:21:19,840
overview it's structured a bit better it

587
00:21:18,400 --> 00:21:23,600
should be much easier to get started

588
00:21:19,840 --> 00:21:26,240
with k unit um and a bit easier to find

589
00:21:23,600 --> 00:21:27,919
specifics on how do i write tests versus

590
00:21:26,240 --> 00:21:30,159
how do i run them versus how do i use

591
00:21:27,919 --> 00:21:31,280
the k unit tool wrapper or how do i do

592
00:21:30,159 --> 00:21:34,880
it manually

593
00:21:31,280 --> 00:21:36,240
and and bug and usability fixes

594
00:21:34,880 --> 00:21:39,120
there are also

595
00:21:36,240 --> 00:21:42,000
a number of better ways of selecting

596
00:21:39,120 --> 00:21:44,080
which tests get run

597
00:21:42,000 --> 00:21:47,280
if you're a maintainer you can now put a

598
00:21:44,080 --> 00:21:49,440
dot k unit config file in a directory of

599
00:21:47,280 --> 00:21:52,799
the kernel source code uh for your

600
00:21:49,440 --> 00:21:54,799
subsystem and list all of the options

601
00:21:52,799 --> 00:21:57,440
that you want k unit to use when testing

602
00:21:54,799 --> 00:21:59,679
that subsystem and people can pass

603
00:21:57,440 --> 00:22:02,720
the path to that k unit config fragment

604
00:21:59,679 --> 00:22:05,039
in the path to that subsystem in um and

605
00:22:02,720 --> 00:22:07,520
the kernel will run

606
00:22:05,039 --> 00:22:09,760
be built with all of those features and

607
00:22:07,520 --> 00:22:12,000
tests included

608
00:22:09,760 --> 00:22:14,000
and will be run so that's a great way of

609
00:22:12,000 --> 00:22:15,280
saying run with this

610
00:22:14,000 --> 00:22:17,440
um

611
00:22:15,280 --> 00:22:20,400
and you can

612
00:22:17,440 --> 00:22:22,640
execute things um

613
00:22:20,400 --> 00:22:26,400
see all of the test results this is what

614
00:22:22,640 --> 00:22:28,000
i expect before you send me a patch

615
00:22:26,400 --> 00:22:30,159
similarly if you just want to add one

616
00:22:28,000 --> 00:22:32,559
extra config option to one of these

617
00:22:30,159 --> 00:22:35,120
there's this k config add option simple

618
00:22:32,559 --> 00:22:37,679
i want to run the ext4 test but i want

619
00:22:35,120 --> 00:22:39,600
to enable ksan or something like that

620
00:22:37,679 --> 00:22:41,360
it's very easy

621
00:22:39,600 --> 00:22:43,200
test filtering you can test filter what

622
00:22:41,360 --> 00:22:45,679
tests run at runtime as well as at

623
00:22:43,200 --> 00:22:48,640
compile time

624
00:22:45,679 --> 00:22:50,000
and that lets you build your kernel once

625
00:22:48,640 --> 00:22:51,280
and run several different tests in

626
00:22:50,000 --> 00:22:52,880
several different

627
00:22:51,280 --> 00:22:54,720
environments with it which can be quite

628
00:22:52,880 --> 00:22:56,320
convenient too and is a prerequisite for

629
00:22:54,720 --> 00:22:59,200
hermetic testing

630
00:22:56,320 --> 00:23:00,640
um we all know that tests in theory

631
00:22:59,200 --> 00:23:02,559
should leave everything the way they

632
00:23:00,640 --> 00:23:04,559
found them and it shouldn't matter what

633
00:23:02,559 --> 00:23:06,799
order you run the tests in because test

634
00:23:04,559 --> 00:23:09,360
a is not going to break test b that runs

635
00:23:06,799 --> 00:23:11,440
afterwards by scribbling over memory or

636
00:23:09,360 --> 00:23:13,039
whatever but equally in the real world

637
00:23:11,440 --> 00:23:13,840
that doesn't always happen

638
00:23:13,039 --> 00:23:16,240
so

639
00:23:13,840 --> 00:23:18,640
to help debug those cases there's a run

640
00:23:16,240 --> 00:23:22,240
isolated option to k unit tool which

641
00:23:18,640 --> 00:23:24,880
will run each test or each test suite in

642
00:23:22,240 --> 00:23:27,360
its own invocation of the kernel

643
00:23:24,880 --> 00:23:30,159
so they can't stamp on each other's toes

644
00:23:27,360 --> 00:23:33,039
very useful for tracking down weird bugs

645
00:23:30,159 --> 00:23:35,360
where memory has been corrupted or the

646
00:23:33,039 --> 00:23:35,360
like

647
00:23:35,840 --> 00:23:39,840
finally there are a bunch of new tests

648
00:23:37,440 --> 00:23:41,200
this is always my favorite slide um

649
00:23:39,840 --> 00:23:43,360
in the last year we've doubled the

650
00:23:41,200 --> 00:23:46,880
number of test suites in cayun and 511

651
00:23:43,360 --> 00:23:51,039
we had 20 and 516 there are 40 um

652
00:23:46,880 --> 00:23:52,720
from 204 individual tests to 324

653
00:23:51,039 --> 00:23:54,559
and this is despite the fact that the

654
00:23:52,720 --> 00:23:56,240
parameterized test feature that was

655
00:23:54,559 --> 00:23:59,240
merged at the end of

656
00:23:56,240 --> 00:23:59,240
2020

657
00:23:59,520 --> 00:24:03,919
now has

658
00:24:01,600 --> 00:24:05,360
basically merged a bunch of other tests

659
00:24:03,919 --> 00:24:08,000
so

660
00:24:05,360 --> 00:24:09,120
the number went down before it went up

661
00:24:08,000 --> 00:24:10,960
but

662
00:24:09,120 --> 00:24:12,960
some of these new tests are really neat

663
00:24:10,960 --> 00:24:14,320
uh one of them i i won't go through the

664
00:24:12,960 --> 00:24:16,320
whole list but one of them i want to

665
00:24:14,320 --> 00:24:18,400
point out is these timestamp conversion

666
00:24:16,320 --> 00:24:22,000
tests for

667
00:24:18,400 --> 00:24:23,440
converting from unix time to civil time

668
00:24:22,000 --> 00:24:26,400
there it turns out someone's written

669
00:24:23,440 --> 00:24:29,120
this paper of how you can use this

670
00:24:26,400 --> 00:24:31,360
frankly terrifying bit of algebra to do

671
00:24:29,120 --> 00:24:32,799
this really efficiently

672
00:24:31,360 --> 00:24:34,960
and in order to introduce that to the

673
00:24:32,799 --> 00:24:36,880
kernel um they used test driven

674
00:24:34,960 --> 00:24:38,159
development by writing k unit tests for

675
00:24:36,880 --> 00:24:39,120
the existing

676
00:24:38,159 --> 00:24:40,799
uh

677
00:24:39,120 --> 00:24:41,919
you know naive implementation of the

678
00:24:40,799 --> 00:24:43,840
conversion

679
00:24:41,919 --> 00:24:46,480
then implemented their their new

680
00:24:43,840 --> 00:24:48,320
algorithm and then checked that the

681
00:24:46,480 --> 00:24:49,760
tests still passed

682
00:24:48,320 --> 00:24:50,720
and that's really neat they've done that

683
00:24:49,760 --> 00:24:52,640
for both

684
00:24:50,720 --> 00:24:54,880
regular timestamp conversions and rtc

685
00:24:52,640 --> 00:24:56,960
clock conversions um

686
00:24:54,880 --> 00:24:58,400
k-fence is a new

687
00:24:56,960 --> 00:25:00,880
memory checker

688
00:24:58,400 --> 00:25:03,440
there's sound driver sd card driver

689
00:25:00,880 --> 00:25:06,080
thunderbolt multipath tcp

690
00:25:03,440 --> 00:25:08,799
stack unwinding for the s390

691
00:25:06,080 --> 00:25:10,559
memory management daemon's really cool

692
00:25:08,799 --> 00:25:12,720
uh this was the first

693
00:25:10,559 --> 00:25:14,240
project that actually used k unit and

694
00:25:12,720 --> 00:25:15,840
it's finally upstream so we're really

695
00:25:14,240 --> 00:25:18,000
excited

696
00:25:15,840 --> 00:25:20,080
basic kernel functions and c functions

697
00:25:18,000 --> 00:25:23,600
like mem set mem copy mem move all

698
00:25:20,080 --> 00:25:27,120
actually have some k unit tests um

699
00:25:23,600 --> 00:25:28,400
the hashing for files it's all there

700
00:25:27,120 --> 00:25:30,640
so

701
00:25:28,400 --> 00:25:32,320
what's what's coming up in the future

702
00:25:30,640 --> 00:25:33,919
um

703
00:25:32,320 --> 00:25:35,679
we're going to fix all of the things to

704
00:25:33,919 --> 00:25:37,919
actually match this new ktap standard

705
00:25:35,679 --> 00:25:39,760
that we wrote uh we started on that but

706
00:25:37,919 --> 00:25:41,200
there are still obviously a number of

707
00:25:39,760 --> 00:25:43,279
things that now we know what the

708
00:25:41,200 --> 00:25:44,640
standard is we can bring up to it

709
00:25:43,279 --> 00:25:46,880
um

710
00:25:44,640 --> 00:25:49,679
improved support for running tests as

711
00:25:46,880 --> 00:25:52,320
modules is a big one

712
00:25:49,679 --> 00:25:54,799
as k unit

713
00:25:52,320 --> 00:25:56,320
developers we mostly build the tests

714
00:25:54,799 --> 00:25:58,080
into the kernel image rather than

715
00:25:56,320 --> 00:26:00,320
loading them as modules afterwards but k

716
00:25:58,080 --> 00:26:03,039
unit does support this

717
00:26:00,320 --> 00:26:05,600
and jeremy who as i think is here has

718
00:26:03,039 --> 00:26:07,760
started uh proposing this new way of

719
00:26:05,600 --> 00:26:09,120
handling modules that is a lot less

720
00:26:07,760 --> 00:26:11,279
weirdly uh

721
00:26:09,120 --> 00:26:13,520
hacky than our old ones so we're looking

722
00:26:11,279 --> 00:26:15,600
forward to improving that

723
00:26:13,520 --> 00:26:17,840
more test test examples particularly for

724
00:26:15,600 --> 00:26:19,600
testing hardware

725
00:26:17,840 --> 00:26:22,159
we've reduced memory usage and stack

726
00:26:19,600 --> 00:26:24,320
size even further

727
00:26:22,159 --> 00:26:26,960
which is is particularly useful since

728
00:26:24,320 --> 00:26:29,760
linus was whinging about it uh when w

729
00:26:26,960 --> 00:26:32,960
error went in and a number of uh bug

730
00:26:29,760 --> 00:26:35,279
fixes and documentation changes

731
00:26:32,960 --> 00:26:35,279
um

732
00:26:35,520 --> 00:26:41,440
the final thing is please reach out if

733
00:26:38,000 --> 00:26:43,039
you have any um questions for us i'm

734
00:26:41,440 --> 00:26:45,600
very curious if people have actually

735
00:26:43,039 --> 00:26:47,200
been using ku nor k self-tested

736
00:26:45,600 --> 00:26:49,600
if there's anything blocking people from

737
00:26:47,200 --> 00:26:50,640
doing so

738
00:26:49,600 --> 00:26:53,039
whether

739
00:26:50,640 --> 00:26:55,039
maintainers in particular wants to have

740
00:26:53,039 --> 00:26:57,440
lists of tests they want people to run

741
00:26:55,039 --> 00:26:59,360
using this k unit config mechanism or

742
00:26:57,440 --> 00:27:01,600
another like an entry in maintainers

743
00:26:59,360 --> 00:27:02,799
saying please run these before you send

744
00:27:01,600 --> 00:27:05,039
me patches

745
00:27:02,799 --> 00:27:06,640
and the contributors like the idea of

746
00:27:05,039 --> 00:27:08,240
that

747
00:27:06,640 --> 00:27:10,080
if you send a patch will you tell people

748
00:27:08,240 --> 00:27:12,400
what you've tested in it

749
00:27:10,080 --> 00:27:13,840
and how much refactoring of code i think

750
00:27:12,400 --> 00:27:15,520
this is going to be the theme for the

751
00:27:13,840 --> 00:27:17,279
next year

752
00:27:15,520 --> 00:27:18,799
is acceptable to make something more

753
00:27:17,279 --> 00:27:21,120
testable

754
00:27:18,799 --> 00:27:22,799
because a lot of code in the kernel is

755
00:27:21,120 --> 00:27:25,600
intertwined in a way

756
00:27:22,799 --> 00:27:26,799
um that makes it difficult

757
00:27:25,600 --> 00:27:29,360
okay

758
00:27:26,799 --> 00:27:31,600
uh questions comments

759
00:27:29,360 --> 00:27:32,799
uh i see there are a few questions here

760
00:27:31,600 --> 00:27:35,520
um

761
00:27:32,799 --> 00:27:37,360
so i'll get started um the first one

762
00:27:35,520 --> 00:27:38,960
from andrew tap wise are we going to be

763
00:27:37,360 --> 00:27:40,799
okay with reporting multiple sets of

764
00:27:38,960 --> 00:27:42,880
results where kernel tests may be run at

765
00:27:40,799 --> 00:27:45,200
different times like initialization

766
00:27:42,880 --> 00:27:47,760
versus device pro versus module

767
00:27:45,200 --> 00:27:50,320
in it um

768
00:27:47,760 --> 00:27:52,159
this basically is is one of the tricky

769
00:27:50,320 --> 00:27:55,200
things um

770
00:27:52,159 --> 00:27:56,240
k unit uh tooling we're hoping to make

771
00:27:55,200 --> 00:27:57,919
handle

772
00:27:56,240 --> 00:27:59,760
okay if there are multiple sets of

773
00:27:57,919 --> 00:28:03,039
results we'll be able to merge them uh

774
00:27:59,760 --> 00:28:05,600
it's not quite done yet but uh you know

775
00:28:03,039 --> 00:28:07,919
in theory it should be able to work

776
00:28:05,600 --> 00:28:08,720
um

777
00:28:07,919 --> 00:28:10,399
uh

778
00:28:08,720 --> 00:28:12,720
most tests

779
00:28:10,399 --> 00:28:15,679
you know when you run k unit run in it

780
00:28:12,720 --> 00:28:18,000
um and there's this uh dev

781
00:28:15,679 --> 00:28:19,679
debug fs way of getting the individual

782
00:28:18,000 --> 00:28:22,880
results for

783
00:28:19,679 --> 00:28:24,640
things that are loaded later

784
00:28:22,880 --> 00:28:26,960
do any cis run

785
00:28:24,640 --> 00:28:28,000
kernel tests with random enabled config

786
00:28:26,960 --> 00:28:30,880
options

787
00:28:28,000 --> 00:28:32,880
um i know there are some that at least

788
00:28:30,880 --> 00:28:34,000
build them with random enabled config

789
00:28:32,880 --> 00:28:35,279
options

790
00:28:34,000 --> 00:28:37,200
um

791
00:28:35,279 --> 00:28:38,480
i'm not sure off the top of my head if

792
00:28:37,200 --> 00:28:40,320
any of them running there are some that

793
00:28:38,480 --> 00:28:41,760
run with um

794
00:28:40,320 --> 00:28:44,399
i think there are there are certainly

795
00:28:41,760 --> 00:28:47,039
some that run with with pretty large uh

796
00:28:44,399 --> 00:28:48,960
all yes config style setups

797
00:28:47,039 --> 00:28:51,279
um

798
00:28:48,960 --> 00:28:52,720
uh the hermetic testing

799
00:28:51,279 --> 00:28:55,039
um

800
00:28:52,720 --> 00:28:57,200
can the hermetic testing run isolated

801
00:28:55,039 --> 00:28:59,120
feature uh be used for target specific

802
00:28:57,200 --> 00:29:01,279
drivers tests only

803
00:28:59,120 --> 00:29:05,440
eg when developing an out of tree

804
00:29:01,279 --> 00:29:05,440
prototype driver against a stable kernel

805
00:29:06,080 --> 00:29:11,039
at the moment run isolated as a feature

806
00:29:08,559 --> 00:29:13,600
can't be used to separate one driver

807
00:29:11,039 --> 00:29:15,919
from the others in the same execution

808
00:29:13,600 --> 00:29:19,039
but you can combine it

809
00:29:15,919 --> 00:29:23,679
or use instead the uh glob filter thing

810
00:29:19,039 --> 00:29:25,520
to run just a particular driver um

811
00:29:23,679 --> 00:29:26,799
with a separate invocation of k unit

812
00:29:25,520 --> 00:29:27,760
tool

813
00:29:26,799 --> 00:29:30,720
um

814
00:29:27,760 --> 00:29:33,919
so it's definitely possible to do that

815
00:29:30,720 --> 00:29:35,919
um you just you know if if the driver

816
00:29:33,919 --> 00:29:37,039
was called foo you can just k unit dot

817
00:29:35,919 --> 00:29:39,440
pi run

818
00:29:37,039 --> 00:29:40,799
foo star and it will run all tests that

819
00:29:39,440 --> 00:29:41,760
start with foo

820
00:29:40,799 --> 00:29:45,520
so

821
00:29:41,760 --> 00:29:45,520
if you name things well it's easy

822
00:29:46,799 --> 00:29:51,039
well thank you very much david that

823
00:29:48,960 --> 00:29:53,520
brings us to the end of our time flop

824
00:29:51,039 --> 00:29:55,840
for this talk

825
00:29:53,520 --> 00:29:58,640
and david will be uh more than happy to

826
00:29:55,840 --> 00:30:00,159
take further questions or comments uh

827
00:29:58,640 --> 00:30:04,320
outside of the session

828
00:30:00,159 --> 00:30:06,399
um yes so thank you thank you very much

829
00:30:04,320 --> 00:30:09,600
um we'll now take a another 10 minute

830
00:30:06,399 --> 00:30:12,159
break and we'll be back at 5 p.m for

831
00:30:09,600 --> 00:30:14,159
russell curry abusing github action for

832
00:30:12,159 --> 00:30:17,159
building and testing kernels see you all

833
00:30:14,159 --> 00:30:17,159
then