aboutsummaryrefslogtreecommitdiff
path: root/Documentation/admin-guide/dynamic-debug-howto.rst
AgeCommit message (Collapse)Author
2021-10-21dyndbg: refine verbosity 1-4 summary-detailJim Cromie
adjust current v*pr_info() calls to fit an overview..detail scheme: 1- module level activity: add/remove, etc 2- command ingest, splitting, summary of effects. per >control write 3- command parsing: op, flags, search terms 4- per-site change msg can yield ~3k x 2 logs per echo "+p;-p" > command. Summarize these 4 levels in MODULE_PARM_DESC, and update verbose=3 in Doc. 2- is new, to isolate a problem where a stress-test script (which feeds ~4kb multi-command strings) would produce short writes, truncating last command and causing parsing errors, which confused test results. The script fix was to use syswrite, to deliver full proper commands. 4- gets per-callsite "changed:" pr-infos, which are very noisy during stress tests, and formerly obscured v1-3 messages, and overwhelmed the static-key workload being tested. The verbose parameter has previously seen adjustment: commit 481c0e33f1e7 ("dyndbg: refine debug verbosity; 1 is basic, 2 more chatty") The script driving these adjustments is: !/usr/bin/perl -w =for Doc 1st purpose was to benchmark the effect of wildcard queries on query performance; if wildcards are risk free cheap enough, we can deploy them in the (floating) format search. 1st finding: wildcards take 2x as long to process. 2nd purpose was to benchmark real static-key changes VS simple flag changes. Found ~100x decrease for the hard work. The script maximizes workload per >control by packing it a ~4kb string of "+p; -p;" commands; this uncovered some broken stuff. The 85th query failed, and appears to be truncated, so is gramatically incorrect. Its either an error here, or in the kernel. Its not happening atm, retest. Plot thickens: fail only happens doing +-p, not +-mf, likely load dependent. Error remains consistent. Looks like a short write, longer on writer than kernel-reader. Try syswrite on handle to control this. That fixed short write. =cut use Getopt::Std; getopts('vN:k:', \my %opts) or die <<EOH; $0 options: -v verbose -k=n kernel dyndbg verbosity -N=n number of loops.. tbrc EOH $opts{N} //= 10; # !undef, 0 tests too long. my $ctrl = '/proc/dynamic_debug/control'; vx($opts{k}) if defined $opts{k}; # works on -k0 open(my $CTL, '>', $ctrl) or die "cant open $ctrl for writing: $!\n"; sub vx { my $arg = shift; my $cmd = "echo $arg > /sys/module/dynamic_debug/parameters/verbose"; system($cmd); warn("vx problem: rc:$? err:$! qry: $cmd\n") if ($?); } sub qryOK { my $qry = shift; print "syntax test: <\n$qry>\n" if $opts{v}; my $bytes = syswrite $CTL, $qry; printf "short read: $bytes / %d\n", length $qry if $bytes < length $qry; if ($?) { warn "rc:$? err:$! qry: $qry\n"; return 0; } return 1; } sub build_queries { my ($cmd, $flags, $ct) = @_; # build experiment and reference queries my $cycle = " $cmd +$flags # on ; $cmd -$flags # off \n"; my $ref = " +$flags ; -$flags \n"; my $len = length $cycle; my $max = int(4096 / $len); # break/fit to buffer size $ct |= $max; print "qry: ct:$max x << \n$cycle >>\n"; return unless qryOK($ref); return unless qryOK($cycle); my $wild = $cycle x $ct; my $empty = $ref x $ct; printf "len: %d, %d\n", length $wild, length $empty; return { trial => $wild, ref => $empty, probe => $cycle, zero => $ref, count => $ct, max => $max }; } my $query_set = build_queries(' file "*" module "*" func "*" ', "mf"); qryOK($query_set->{zero}); qryOK($query_set->{probe}); qryOK($query_set->{ref}); qryOK($query_set->{trial}); use Benchmark; sub dobatch { my ($cmd, $flags, $reps, $ct) = @_; $reps ||= $opts{N}; my $qs = build_queries($cmd, $flags, $ct); timethese($reps, { wildcards => sub { syswrite $CTL, $qs->{trial}; }, no_search => sub { syswrite $CTL, $qs->{ref}; } } ); } sub bench_static_key_toggle { vx 0; dobatch(' file "*" module "*" func "*" ', "mf"); dobatch(' file "*" module "*" func "*" ', "p"); } sub bench_verbose_levels { for my $i (0..4) { vx $i; dobatch(' file "*" module "*" func "*" ', "mf"); } } bench_static_key_toggle(); __END__ Heres how the test-script runs: :: verbose=3 parsing info [ 48.401646] dyndbg: query 95: "file "*" module "*" func "*" -mf # off " mod:* [ 48.402040] dyndbg: split into words: "file" "*" "module" "*" "func" "*" "-mf" [ 48.402456] dyndbg: op='-' [ 48.402615] dyndbg: flags=0x6 [ 48.402779] dyndbg: *flagsp=0x0 *maskp=0xfffffff9 [ 48.403033] dyndbg: parsed: func="*" file="*" module="*" format="" lineno=0-0 [ 48.403674] dyndbg: applied: func="*" file="*" module="*" format="" lineno=0-0 :: verbose=2 >control summary. ~300k site matches/changes per 4kb command [ 48.404063] dyndbg: processed 96 queries, with 296160 matches, 0 errs :: 2 queries against each other, no-search vs all-wildcard-search qry: ct:48 x << file "*" module "*" func "*" +mf # on ; file "*" module "*" func "*" -mf # off >> len: 4080, 576 Benchmark: timing 10 iterations of no_search, wildcards... no_search: 0 wallclock secs ( 0.00 usr + 0.03 sys = 0.03 CPU) @ 333.33/s (n=10) (warning: too few iterations for a reliable count) wildcards: 0 wallclock secs ( 0.00 usr + 0.09 sys = 0.09 CPU) @ 111.11/s (n=10) (warning: too few iterations for a reliable count) :: 2 queries, both doing real work / changing stati-key states. qry: ct:49 x << file "*" module "*" func "*" +p # on ; file "*" module "*" func "*" -p # off >> len: 4067, 490 Benchmark: timing 10 iterations of no_search, wildcards... no_search: 20 wallclock secs ( 0.00 usr + 20.36 sys = 20.36 CPU) @ 0.49/s (n=10) wildcards: 21 wallclock secs ( 0.00 usr + 21.08 sys = 21.08 CPU) @ 0.47/s (n=10) bash-5.1# Thats 150k static-key-toggles / sec ~600x slower than simple flags on qemu --smp 3 run Signed-off-by: Jim Cromie <jim.cromie@gmail.com> Link: https://lore.kernel.org/r/20211019210746.185307-1-jim.cromie@gmail.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-10-14Documentation: dyndbg: Improve cli param examplesAndrew Halaney
Jim pointed out that using $module.dyndbg= is always a more flexible choice for using dynamic debug on the command line. The $module.dyndbg style is checked at boot and handles if $module is a builtin. If it is actually a loadable module, it is handled again later when the module is loaded. If you just use dyndbg="module $module +p" dynamic debug is only enabled when $module is a builtin. It was recommended to illustrate wildcard usage as well. Suggested-by: Jim Cromie <jim.cromie@gmail.com> Signed-off-by: Andrew Halaney <ahalaney@redhat.com> Signed-off-by: Jason Baron <jbaron@akamai.com> Link: https://lore.kernel.org/r/1634139622-20667-4-git-send-email-jbaron@akamai.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-10-14dyndbg: Remove support for ddebug_query paramAndrew Halaney
This param has been deprecated for a very long time now, let's rip it out. Signed-off-by: Andrew Halaney <ahalaney@redhat.com> Signed-off-by: Jason Baron <jbaron@akamai.com> Link: https://lore.kernel.org/r/1634139622-20667-3-git-send-email-jbaron@akamai.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-03-08Documentation: dynamic-debug-howto: fix exampleMartin Kepplinger
dynamic debug is "expecting pairs of match-spec <value>" so the example for all files of which the paths include "usb" there is "file" missing. Signed-off-by: Martin Kepplinger <martin.kepplinger@puri.sm> Link: https://lore.kernel.org/r/20210303091646.773111-1-martin.kepplinger@puri.sm Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2020-09-10Revert "dyndbg: accept query terms like file=bar and module=foo"Greg Kroah-Hartman
This reverts commit 14775b04964264189caa4a0862eac05dab8c0502 as there were still some parsing problems with it, and the follow-on patch for it. Let's revisit it later, just drop it for now. Cc: <jbaron@akamai.com> Cc: Jim Cromie <jim.cromie@gmail.com> Reported-by: Naresh Kamboju <naresh.kamboju@linaro.org> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Fixes: 14775b049642 ("dyndbg: accept query terms like file=bar and module=foo") Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2020-07-24dyndbg: accept query terms like file=bar and module=fooJim Cromie
Current code expects "keyword" "arg" as 2 words, space separated. Change to also accept "keyword=arg" form as well, and drop !(nwords%2) requirement. Then in rest of function, use new keyword, arg variables instead of word[i], word[i+1] Acked-by: <jbaron@akamai.com> Signed-off-by: Jim Cromie <jim.cromie@gmail.com> Link: https://lore.kernel.org/r/20200719231058.1586423-15-jim.cromie@gmail.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2020-07-24dyndbg: accept 'file foo.c:func1' and 'file foo.c:10-100'Jim Cromie
Accept these additional query forms: echo "file $filestr +_" > control path/to/file.c:100 # as from control, column 1 path/to/file.c:1-100 # or any legal line-range path/to/file.c:func_A # as from an editor/browser path/to/file.c:drm_* # wildcards still work path/to/file.c:*_foo # lead wildcard too 1st 2 examples are treated as line-ranges, 3-5 are treated as func's Doc these changes, and sprinkle in a few extra wild-card examples and trailing # explanation texts. Acked-by: <jbaron@akamai.com> Signed-off-by: Jim Cromie <jim.cromie@gmail.com> Link: https://lore.kernel.org/r/20200719231058.1586423-14-jim.cromie@gmail.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2020-07-24dyndbg-docs: initialization is done early, not archJim Cromie
since cf964976484 in 2012, initialization is done with early_initcall, update the Docs, which still say arch_initcall. Acked-by: <jbaron@akamai.com> Signed-off-by: Jim Cromie <jim.cromie@gmail.com> Link: https://lore.kernel.org/r/20200719231058.1586423-3-jim.cromie@gmail.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2020-07-24dyndbg-docs: eschew file /full/path query in docsJim Cromie
Regarding: commit 2b6783191da7 ("dynamic_debug: add trim_prefix() to provide source-root relative paths") commit a73619a845d5 ("kbuild: use -fmacro-prefix-map to make __FILE__ a relative path") 2nd commit broke dynamic-debug's "file $fullpath" query form, but nobody noticed because 1st commit had trimmed prefixes from control-file output, so the click-copy-pasting of fullpaths into new queries had ceased; that query form became unused. Removing the function is cleanest, but it could be useful in old-compiler corner cases, where __FILE__ still has /full/path, and it safely does nothing otherwize. So instead, quietly deprecate "file /full/path" query form, by removing all /full/paths examples in the docs. I skipped adding a back-compat note. Acked-by: <jbaron@akamai.com> Signed-off-by: Jim Cromie <jim.cromie@gmail.com> Link: https://lore.kernel.org/r/20200719231058.1586423-2-jim.cromie@gmail.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2020-06-08dynamic_debug: add an option to enable dynamic debug for modules onlyOrson Zhai
Instead of enabling dynamic debug globally with CONFIG_DYNAMIC_DEBUG, CONFIG_DYNAMIC_DEBUG_CORE will only enable core function of dynamic debug. With the DYNAMIC_DEBUG_MODULE defined for any modules, dynamic debug will be tied to them. This is useful for people who only want to enable dynamic debug for kernel modules without worrying about kernel image size and memory consumption is increasing too much. [orson.zhai@unisoc.com: v2] Link: http://lkml.kernel.org/r/1587408228-10861-1-git-send-email-orson.unisoc@gmail.com Signed-off-by: Orson Zhai <orson.zhai@unisoc.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Acked-by: Petr Mladek <pmladek@suse.com> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Jason Baron <jbaron@akamai.com> Cc: Randy Dunlap <rdunlap@infradead.org> Link: http://lkml.kernel.org/r/1586521984-5890-1-git-send-email-orson.unisoc@gmail.com Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-02-12dynamic_debug: allow to work if debugfs is disabledGreg Kroah-Hartman
With the realization that having debugfs enabled on "production" systems is generally not a good idea, debugfs is being disabled from more and more platforms over time. However, the functionality of dynamic debugging still is needed at times, and since it relies on debugfs for its user api, having debugfs disabled also forces dynamic debug to be disabled. To get around this, also create the "control" file for dynamic_debug in procfs. This allows people turn on debugging as needed at runtime for individual driverfs and subsystems. Reported-by: many different companies Cc: Jason Baron <jbaron@akamai.com> Acked-by: Will Deacon <will@kernel.org> Link: https://lore.kernel.org/r/20200210211142.GB1373304@kroah.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2018-11-07Documentation: dynamic-debug: fix wildcard descriptionRandy Dunlap
Fix grammar about wildcards and insert a space between sentences. Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Cc: Jason Baron <jbaron@akamai.com> Cc: Jonathan Corbet <corbet@lwn.net> Cc: linux-doc@vger.kernel.org Cc: Will Korteland <will@korte.land> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2018-11-07Documentation: dynamic_debug: fix a couple of typosRandy Dunlap
Fix a few "typos" in dynamic-debug-howto.rst. s/dyndbg_query/ddebug_query/ s/sysfs/debugfs/ Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Cc: Jason Baron <jbaron@akamai.com> Cc: Jonathan Corbet <corbet@lwn.net> Cc: linux-doc@vger.kernel.org Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-11-17dynamic_debug documentation: minor fixesRandy Dunlap
Fix minor typo. Fix missing words in explaining parsing of last line number. Link: http://lkml.kernel.org/r/ebb7ff42-4945-103f-d5b4-f07a6f3343a7@infradead.org Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Cc: Jason Baron <jbaron@akamai.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2017-01-26dynamic-debug-howto: Correct echo -c to -nSteven Price
Two of the example command lines use an argument to echo of "-c" which isn't valid in (most versions of) echo causing these examples to fail. Correct the argument to "-n" which works correctly. Signed-off-by: Steven Price <steven@ecrips.co.uk> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-10-24docs-rst: create an user's manual bookMauro Carvalho Chehab
Place README, REPORTING-BUGS, SecurityBugs and kernel-parameters on an user's manual book. As we'll be numbering the user's manual, remove the manual numbering from SecurityBugs. Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>