Record and Replay Framework

Related tags

Security tools linux debugger gdb reverse-execution
Overview

Overview

rr is a lightweight tool for recording, replaying and debugging execution of applications (trees of processes and threads). Debugging extends gdb with very efficient reverse-execution, which in combination with standard gdb/x86 features like hardware data watchpoints, makes debugging much more fun. More information about the project, including instructions on how to install, run, and build rr, is at https://rr-project.org. The best technical overview is currently the paper Engineering Record And Replay For Deployability: Extended Technical Report.

Or go directly to the installation and building instructions.

Please contribute! Make sure to review the pull request checklist before submitting a pull request.

If you find rr useful, please add a testimonial.

rr development is sponsored by Pernosco and was originated by Mozilla.

System requirements

  • Linux kernel ≥ 3.11 is required (for PTRACE_SETSIGMASK).
  • rr currently requires either:
  • Running in a VM guest is supported, as long as the VM supports virtualization of hardware performance counters. (VMware and KVM are known to work; Xen does not.)
Comments
  • Support AMD Ryzen?

    Support AMD Ryzen?

    Ryzen has a conditional branch counter. I have patches to use it here: https://github.com/mozilla/rr/tree/ryzen

    To make it work reliably I had to increase the skid counter to 1000. That's pretty high, but OK. The patches make the skid size configurable per-architecture so we don't take that hit on Intel.

    With these patches, most tests pass and the rest seem to be intermittent. In one run I get 10 failures out of 2068:

            327 - proc_fds (Failed)
        388 - ptracer_death_multithread_peer-no-syscallbuf (Failed)
        688 - condvar_stress-no-syscallbuf (Failed)
        835 - thread_stress (Failed)
        904 - checkpoint_prctl_name-no-syscallbuf (Failed)
        1362 - proc_fds-32-no-syscallbuf (Failed)
        1679 - async_signal_syscalls2-32 (Failed)
        1722 - condvar_stress-32-no-syscallbuf (Failed)
        1869 - thread_stress-32 (Failed)
        1870 - thread_stress-32-no-syscallbuf (Failed)

    It appears that all these failures are due to intermittent overcounting. In most of them, during recording we seem to have overcounted a few conditional branches in the leadup to some syscall. In the rest, we seem to have overcounted during replay.

    One interesting thing is that most of the syscalls where we detect the overcount are an mprotect (or a syscall following a syscall-buffered mprotect) that followed an mmap. There are two exceptions, one a read syscall and one a write syscall. I need to think about what this might mean.

    opened by rocallahan 275
  • Support AMD Piledriver?

    Support AMD Piledriver?

    I wanted to try rr but it looks like it doesn't support my AMD CPU:

    [FATAL /home/edwin/rr/rr/src/PerfCounters.cc:129:get_cpu_microarch() errno: 0 'Success'] 
 -> CPU 0xf20 unknown.

    The CPU is an AMD FX-8350, which is AFAICT Piledriver microarchitecture, see below for cpuinfo and perf list output. The available performance counter events should be documented in 3.15 Core Performance Counter Events. Is it possible to support rr on this microarchitecture, and what would be the value for struct PmuConfig?

    $ dmesg|grep PMU
[    0.346218] Performance Events: Fam15h core perfctr, AMD PMU driver.
[    0.346626] NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
$ uname -a
Linux debian 4.2.0-0.bpo.1-amd64 #1 SMP Debian 4.2.3-2~bpo8+1 (2015-10-20) x86_64 GNU/Linux

    /proc/cpuinfo for a core:

    processor   : 7
vendor_id   : AuthenticAMD
cpu family  : 21
model       : 2
model name  : AMD FX(tm)-8350 Eight-Core Processor
stepping    : 0
microcode   : 0x6000832
cpu MHz     : 1400.000
cache size  : 2048 KB
physical id : 0
siblings    : 8
core id     : 7
cpu cores   : 4
apicid      : 23
initial apicid  : 7
fpu     : yes
fpu_exception   : yes
cpuid level : 13
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold vmmcall bmi1
bugs        : fxsave_leak sysret_ss_attrs
bogomips    : 8026.66
TLB size    : 1536 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro

    List of pre-defined events (to be used in -e):

    branch-instructions OR branches [Hardware event] branch-misses [Hardware event] cache-misses [Hardware event] cache-references [Hardware event] cpu-cycles OR cycles [Hardware event] instructions [Hardware event] stalled-cycles-backend OR idle-cycles-backend [Hardware event] stalled-cycles-frontend OR idle-cycles-frontend [Hardware event]

    alignment-faults [Software event] context-switches OR cs [Software event] cpu-clock [Software event] cpu-migrations OR migrations [Software event] dummy [Software event] emulation-faults [Software event] major-faults [Software event] minor-faults [Software event] page-faults OR faults [Software event] task-clock [Software event]

    L1-dcache-load-misses [Hardware cache event] L1-dcache-loads [Hardware cache event] L1-dcache-prefetch-misses [Hardware cache event] L1-dcache-prefetches [Hardware cache event] L1-dcache-stores [Hardware cache event] L1-icache-load-misses [Hardware cache event] L1-icache-loads [Hardware cache event] L1-icache-prefetches [Hardware cache event] LLC-load-misses [Hardware cache event] LLC-loads [Hardware cache event] LLC-stores [Hardware cache event] branch-load-misses [Hardware cache event] branch-loads [Hardware cache event] dTLB-load-misses [Hardware cache event] dTLB-loads [Hardware cache event] iTLB-load-misses [Hardware cache event] iTLB-loads [Hardware cache event]

    branch-instructions OR cpu/branch-instructions/ [Kernel PMU event] branch-misses OR cpu/branch-misses/ [Kernel PMU event] cache-misses OR cpu/cache-misses/ [Kernel PMU event] cache-references OR cpu/cache-references/ [Kernel PMU event] cpu-cycles OR cpu/cpu-cycles/ [Kernel PMU event] instructions OR cpu/instructions/ [Kernel PMU event]

    rNNN [Raw hardware event descriptor] cpu/t1=v1[,t2=v2,t3 ...]/modifier [Raw hardware event descriptor](see 'man perf-list' on how to encode it)

    mem:[/len][:access] [Hardware breakpoint]

    block:block_bio_backmerge [Tracepoint event] block:block_bio_bounce [Tracepoint event] block:block_bio_complete [Tracepoint event] block:block_bio_frontmerge [Tracepoint event] block:block_bio_queue [Tracepoint event] block:block_bio_remap [Tracepoint event] block:block_dirty_buffer [Tracepoint event] block:block_getrq [Tracepoint event] block:block_plug [Tracepoint event] block:block_rq_abort [Tracepoint event] block:block_rq_complete [Tracepoint event] block:block_rq_insert [Tracepoint event] block:block_rq_issue [Tracepoint event] block:block_rq_remap [Tracepoint event] block:block_rq_requeue [Tracepoint event] block:block_sleeprq [Tracepoint event] block:block_split [Tracepoint event] block:block_touch_buffer [Tracepoint event] block:block_unplug [Tracepoint event] btrfs:__extent_writepage [Tracepoint event] btrfs:add_delayed_data_ref [Tracepoint event] btrfs:add_delayed_ref_head [Tracepoint event] btrfs:add_delayed_tree_ref [Tracepoint event] btrfs:alloc_extent_state [Tracepoint event] btrfs:btrfs_all_work_done [Tracepoint event] btrfs:btrfs_chunk_alloc [Tracepoint event] btrfs:btrfs_chunk_free [Tracepoint event] btrfs:btrfs_cow_block [Tracepoint event] btrfs:btrfs_failed_cluster_setup [Tracepoint event] btrfs:btrfs_find_cluster [Tracepoint event] btrfs:btrfs_get_extent [Tracepoint event] btrfs:btrfs_inode_evict [Tracepoint event] btrfs:btrfs_inode_new [Tracepoint event] btrfs:btrfs_inode_request [Tracepoint event] btrfs:btrfs_ordered_extent_add [Tracepoint event] btrfs:btrfs_ordered_extent_put [Tracepoint event] btrfs:btrfs_ordered_extent_remove [Tracepoint event] btrfs:btrfs_ordered_extent_start [Tracepoint event] btrfs:btrfs_ordered_sched [Tracepoint event] btrfs:btrfs_reserve_extent [Tracepoint event] btrfs:btrfs_reserve_extent_cluster [Tracepoint event] btrfs:btrfs_reserved_extent_alloc [Tracepoint event] btrfs:btrfs_reserved_extent_free [Tracepoint event] btrfs:btrfs_setup_cluster [Tracepoint event] btrfs:btrfs_space_reservation [Tracepoint event] btrfs:btrfs_sync_file [Tracepoint event] btrfs:btrfs_sync_fs [Tracepoint event] btrfs:btrfs_transaction_commit [Tracepoint event] btrfs:btrfs_work_queued [Tracepoint event] btrfs:btrfs_work_sched [Tracepoint event] btrfs:btrfs_workqueue_alloc [Tracepoint event] btrfs:btrfs_workqueue_destroy [Tracepoint event] btrfs:btrfs_writepage_end_io_hook [Tracepoint event] btrfs:find_free_extent [Tracepoint event] btrfs:free_extent_state [Tracepoint event] btrfs:run_delayed_data_ref [Tracepoint event] btrfs:run_delayed_ref_head [Tracepoint event] btrfs:run_delayed_tree_ref [Tracepoint event] cfg80211:cfg80211_cac_event [Tracepoint event] cfg80211:cfg80211_ch_switch_notify [Tracepoint event] cfg80211:cfg80211_ch_switch_started_notify [Tracepoint event] cfg80211:cfg80211_chandef_dfs_required [Tracepoint event] cfg80211:cfg80211_cqm_pktloss_notify [Tracepoint event] cfg80211:cfg80211_cqm_rssi_notify [Tracepoint event] cfg80211:cfg80211_del_sta [Tracepoint event] cfg80211:cfg80211_ft_event [Tracepoint event] cfg80211:cfg80211_get_bss [Tracepoint event] cfg80211:cfg80211_gtk_rekey_notify [Tracepoint event] cfg80211:cfg80211_ibss_joined [Tracepoint event] cfg80211:cfg80211_inform_bss_width_frame [Tracepoint event] cfg80211:cfg80211_mgmt_tx_status [Tracepoint event] cfg80211:cfg80211_michael_mic_failure [Tracepoint event] cfg80211:cfg80211_new_sta [Tracepoint event] cfg80211:cfg80211_notify_new_peer_candidate [Tracepoint event] cfg80211:cfg80211_pmksa_candidate_notify [Tracepoint event] cfg80211:cfg80211_probe_status [Tracepoint event] cfg80211:cfg80211_radar_event [Tracepoint event] cfg80211:cfg80211_ready_on_channel [Tracepoint event] cfg80211:cfg80211_ready_on_channel_expired [Tracepoint event] cfg80211:cfg80211_reg_can_beacon [Tracepoint event] cfg80211:cfg80211_report_obss_beacon [Tracepoint event] cfg80211:cfg80211_report_wowlan_wakeup [Tracepoint event] cfg80211:cfg80211_return_bool [Tracepoint event] cfg80211:cfg80211_return_bss [Tracepoint event] cfg80211:cfg80211_return_u32 [Tracepoint event] cfg80211:cfg80211_return_uint [Tracepoint event] cfg80211:cfg80211_rx_mgmt [Tracepoint event] cfg80211:cfg80211_rx_mlme_mgmt [Tracepoint event] cfg80211:cfg80211_rx_spurious_frame [Tracepoint event] cfg80211:cfg80211_rx_unexpected_4addr_frame [Tracepoint event] cfg80211:cfg80211_rx_unprot_mlme_mgmt [Tracepoint event] cfg80211:cfg80211_scan_done [Tracepoint event] cfg80211:cfg80211_sched_scan_results [Tracepoint event] cfg80211:cfg80211_sched_scan_stopped [Tracepoint event] cfg80211:cfg80211_send_assoc_timeout [Tracepoint event] cfg80211:cfg80211_send_auth_timeout [Tracepoint event] cfg80211:cfg80211_send_rx_assoc [Tracepoint event] cfg80211:cfg80211_send_rx_auth [Tracepoint event] cfg80211:cfg80211_stop_iface [Tracepoint event] cfg80211:cfg80211_tdls_oper_request [Tracepoint event] cfg80211:cfg80211_tx_mlme_mgmt [Tracepoint event] cfg80211:rdev_add_key [Tracepoint event] cfg80211:rdev_add_mpath [Tracepoint event] cfg80211:rdev_add_station [Tracepoint event] cfg80211:rdev_add_tx_ts [Tracepoint event] cfg80211:rdev_add_virtual_intf [Tracepoint event] cfg80211:rdev_assoc [Tracepoint event] cfg80211:rdev_auth [Tracepoint event] cfg80211:rdev_cancel_remain_on_channel [Tracepoint event] cfg80211:rdev_change_beacon [Tracepoint event] cfg80211:rdev_change_bss [Tracepoint event] cfg80211:rdev_change_mpath [Tracepoint event] cfg80211:rdev_change_station [Tracepoint event] cfg80211:rdev_change_virtual_intf [Tracepoint event] cfg80211:rdev_channel_switch [Tracepoint event] cfg80211:rdev_connect [Tracepoint event] cfg80211:rdev_crit_proto_start [Tracepoint event] cfg80211:rdev_crit_proto_stop [Tracepoint event] cfg80211:rdev_deauth [Tracepoint event] cfg80211:rdev_del_key [Tracepoint event] cfg80211:rdev_del_mpath [Tracepoint event] cfg80211:rdev_del_pmksa [Tracepoint event] cfg80211:rdev_del_station [Tracepoint event] cfg80211:rdev_del_tx_ts [Tracepoint event] cfg80211:rdev_del_virtual_intf [Tracepoint event] cfg80211:rdev_disassoc [Tracepoint event] cfg80211:rdev_disconnect [Tracepoint event] cfg80211:rdev_dump_mpath [Tracepoint event] cfg80211:rdev_dump_mpp [Tracepoint event] cfg80211:rdev_dump_station [Tracepoint event] cfg80211:rdev_dump_survey [Tracepoint event] cfg80211:rdev_flush_pmksa [Tracepoint event] cfg80211:rdev_get_antenna [Tracepoint event] cfg80211:rdev_get_channel [Tracepoint event] cfg80211:rdev_get_key [Tracepoint event] cfg80211:rdev_get_mesh_config [Tracepoint event] cfg80211:rdev_get_mpath [Tracepoint event] cfg80211:rdev_get_mpp [Tracepoint event] cfg80211:rdev_get_station [Tracepoint event] cfg80211:rdev_get_tx_power [Tracepoint event] cfg80211:rdev_join_ibss [Tracepoint event] cfg80211:rdev_join_mesh [Tracepoint event] cfg80211:rdev_join_ocb [Tracepoint event] cfg80211:rdev_leave_ibss [Tracepoint event] cfg80211:rdev_leave_mesh [Tracepoint event] cfg80211:rdev_leave_ocb [Tracepoint event] cfg80211:rdev_libertas_set_mesh_channel [Tracepoint event] cfg80211:rdev_mgmt_frame_register [Tracepoint event] cfg80211:rdev_mgmt_tx [Tracepoint event] cfg80211:rdev_mgmt_tx_cancel_wait [Tracepoint event] cfg80211:rdev_probe_client [Tracepoint event] cfg80211:rdev_remain_on_channel [Tracepoint event] cfg80211:rdev_resume [Tracepoint event] cfg80211:rdev_return_chandef [Tracepoint event] cfg80211:rdev_return_int [Tracepoint event] cfg80211:rdev_return_int_cookie [Tracepoint event] cfg80211:rdev_return_int_int [Tracepoint event] cfg80211:rdev_return_int_mesh_config [Tracepoint event] cfg80211:rdev_return_int_mpath_info [Tracepoint event] cfg80211:rdev_return_int_station_info [Tracepoint event] cfg80211:rdev_return_int_survey_info [Tracepoint event] cfg80211:rdev_return_int_tx_rx [Tracepoint event] cfg80211:rdev_return_void [Tracepoint event] cfg80211:rdev_return_void_tx_rx [Tracepoint event] cfg80211:rdev_return_wdev [Tracepoint event] cfg80211:rdev_rfkill_poll [Tracepoint event] cfg80211:rdev_scan [Tracepoint event] cfg80211:rdev_sched_scan_start [Tracepoint event] cfg80211:rdev_sched_scan_stop [Tracepoint event] cfg80211:rdev_set_antenna [Tracepoint event] cfg80211:rdev_set_ap_chanwidth [Tracepoint event] cfg80211:rdev_set_bitrate_mask [Tracepoint event] cfg80211:rdev_set_cqm_rssi_config [Tracepoint event] cfg80211:rdev_set_cqm_txe_config [Tracepoint event] cfg80211:rdev_set_default_key [Tracepoint event] cfg80211:rdev_set_default_mgmt_key [Tracepoint event] cfg80211:rdev_set_mac_acl [Tracepoint event] cfg80211:rdev_set_monitor_channel [Tracepoint event] cfg80211:rdev_set_noack_map [Tracepoint event] cfg80211:rdev_set_pmksa [Tracepoint event] cfg80211:rdev_set_power_mgmt [Tracepoint event] cfg80211:rdev_set_qos_map [Tracepoint event] cfg80211:rdev_set_rekey_data [Tracepoint event] cfg80211:rdev_set_tx_power [Tracepoint event] cfg80211:rdev_set_txq_params [Tracepoint event] cfg80211:rdev_set_wakeup [Tracepoint event] cfg80211:rdev_set_wds_peer [Tracepoint event] cfg80211:rdev_set_wiphy_params [Tracepoint event] cfg80211:rdev_start_ap [Tracepoint event] cfg80211:rdev_start_p2p_device [Tracepoint event] cfg80211:rdev_stop_ap [Tracepoint event] cfg80211:rdev_stop_p2p_device [Tracepoint event] cfg80211:rdev_suspend [Tracepoint event] cfg80211:rdev_tdls_cancel_channel_switch [Tracepoint event] cfg80211:rdev_tdls_channel_switch [Tracepoint event] cfg80211:rdev_tdls_mgmt [Tracepoint event] cfg80211:rdev_tdls_oper [Tracepoint event] cfg80211:rdev_update_ft_ies [Tracepoint event] cfg80211:rdev_update_mesh_config [Tracepoint event] clk:clk_disable [Tracepoint event] clk:clk_disable_complete [Tracepoint event] clk:clk_enable [Tracepoint event] clk:clk_enable_complete [Tracepoint event] clk:clk_prepare [Tracepoint event] clk:clk_prepare_complete [Tracepoint event] clk:clk_set_parent [Tracepoint event] clk:clk_set_parent_complete [Tracepoint event] clk:clk_set_phase [Tracepoint event] clk:clk_set_phase_complete [Tracepoint event] clk:clk_set_rate [Tracepoint event] clk:clk_set_rate_complete [Tracepoint event] clk:clk_unprepare [Tracepoint event] clk:clk_unprepare_complete [Tracepoint event] compaction:mm_compaction_begin [Tracepoint event] compaction:mm_compaction_defer_compaction [Tracepoint event] compaction:mm_compaction_defer_reset [Tracepoint event] compaction:mm_compaction_deferred [Tracepoint event] compaction:mm_compaction_end [Tracepoint event] compaction:mm_compaction_finished [Tracepoint event] compaction:mm_compaction_isolate_freepages [Tracepoint event] compaction:mm_compaction_isolate_migratepages [Tracepoint event] compaction:mm_compaction_migratepages [Tracepoint event] compaction:mm_compaction_suitable [Tracepoint event] compaction:mm_compaction_try_to_compact_pages [Tracepoint event] drm:drm_vblank_event [Tracepoint event] drm:drm_vblank_event_delivered [Tracepoint event] drm:drm_vblank_event_queued [Tracepoint event] exceptions:page_fault_kernel [Tracepoint event] exceptions:page_fault_user [Tracepoint event] ext4:ext4_alloc_da_blocks [Tracepoint event] ext4:ext4_allocate_blocks [Tracepoint event] ext4:ext4_allocate_inode [Tracepoint event] ext4:ext4_begin_ordered_truncate [Tracepoint event] ext4:ext4_collapse_range [Tracepoint event] ext4:ext4_da_release_space [Tracepoint event] ext4:ext4_da_reserve_space [Tracepoint event] ext4:ext4_da_update_reserve_space [Tracepoint event] ext4:ext4_da_write_begin [Tracepoint event] ext4:ext4_da_write_end [Tracepoint event] ext4:ext4_da_write_pages [Tracepoint event] ext4:ext4_da_write_pages_extent [Tracepoint event] ext4:ext4_direct_IO_enter [Tracepoint event] ext4:ext4_direct_IO_exit [Tracepoint event] ext4:ext4_discard_blocks [Tracepoint event] ext4:ext4_discard_preallocations [Tracepoint event] ext4:ext4_drop_inode [Tracepoint event] ext4:ext4_es_cache_extent [Tracepoint event] ext4:ext4_es_find_delayed_extent_range_enter [Tracepoint event] ext4:ext4_es_find_delayed_extent_range_exit [Tracepoint event] ext4:ext4_es_insert_extent [Tracepoint event] ext4:ext4_es_lookup_extent_enter [Tracepoint event] ext4:ext4_es_lookup_extent_exit [Tracepoint event] ext4:ext4_es_remove_extent [Tracepoint event] ext4:ext4_es_shrink [Tracepoint event] ext4:ext4_es_shrink_count [Tracepoint event] ext4:ext4_es_shrink_scan_enter [Tracepoint event] ext4:ext4_es_shrink_scan_exit [Tracepoint event] ext4:ext4_evict_inode [Tracepoint event] ext4:ext4_ext_convert_to_initialized_enter [Tracepoint event] ext4:ext4_ext_convert_to_initialized_fastpath [Tracepoint event] ext4:ext4_ext_handle_unwritten_extents [Tracepoint event] ext4:ext4_ext_in_cache [Tracepoint event] ext4:ext4_ext_load_extent [Tracepoint event] ext4:ext4_ext_map_blocks_enter [Tracepoint event] ext4:ext4_ext_map_blocks_exit [Tracepoint event] ext4:ext4_ext_put_in_cache [Tracepoint event] ext4:ext4_ext_remove_space [Tracepoint event] ext4:ext4_ext_remove_space_done [Tracepoint event] ext4:ext4_ext_rm_idx [Tracepoint event] ext4:ext4_ext_rm_leaf [Tracepoint event] ext4:ext4_ext_show_extent [Tracepoint event] ext4:ext4_fallocate_enter [Tracepoint event] ext4:ext4_fallocate_exit [Tracepoint event] ext4:ext4_find_delalloc_range [Tracepoint event] ext4:ext4_forget [Tracepoint event] ext4:ext4_free_blocks [Tracepoint event] ext4:ext4_free_inode [Tracepoint event] ext4:ext4_get_implied_cluster_alloc_exit [Tracepoint event] ext4:ext4_get_reserved_cluster_alloc [Tracepoint event] ext4:ext4_ind_map_blocks_enter [Tracepoint event] ext4:ext4_ind_map_blocks_exit [Tracepoint event] ext4:ext4_insert_range [Tracepoint event] ext4:ext4_invalidatepage [Tracepoint event] ext4:ext4_journal_start [Tracepoint event] ext4:ext4_journal_start_reserved [Tracepoint event] ext4:ext4_journalled_invalidatepage [Tracepoint event] ext4:ext4_journalled_write_end [Tracepoint event] ext4:ext4_load_inode [Tracepoint event] ext4:ext4_load_inode_bitmap [Tracepoint event] ext4:ext4_mark_inode_dirty [Tracepoint event] ext4:ext4_mb_bitmap_load [Tracepoint event] ext4:ext4_mb_buddy_bitmap_load [Tracepoint event] ext4:ext4_mb_discard_preallocations [Tracepoint event] ext4:ext4_mb_new_group_pa [Tracepoint event] ext4:ext4_mb_new_inode_pa [Tracepoint event] ext4:ext4_mb_release_group_pa [Tracepoint event] ext4:ext4_mb_release_inode_pa [Tracepoint event] ext4:ext4_mballoc_alloc [Tracepoint event] ext4:ext4_mballoc_discard [Tracepoint event] ext4:ext4_mballoc_free [Tracepoint event] ext4:ext4_mballoc_prealloc [Tracepoint event] ext4:ext4_other_inode_update_time [Tracepoint event] ext4:ext4_punch_hole [Tracepoint event] ext4:ext4_read_block_bitmap_load [Tracepoint event] ext4:ext4_readpage [Tracepoint event] ext4:ext4_releasepage [Tracepoint event] ext4:ext4_remove_blocks [Tracepoint event] ext4:ext4_request_blocks [Tracepoint event] ext4:ext4_request_inode [Tracepoint event] ext4:ext4_sync_file_enter [Tracepoint event] ext4:ext4_sync_file_exit [Tracepoint event] ext4:ext4_sync_fs [Tracepoint event] ext4:ext4_trim_all_free [Tracepoint event] ext4:ext4_trim_extent [Tracepoint event] ext4:ext4_truncate_enter [Tracepoint event] ext4:ext4_truncate_exit [Tracepoint event] ext4:ext4_unlink_enter [Tracepoint event] ext4:ext4_unlink_exit [Tracepoint event] ext4:ext4_write_begin [Tracepoint event] ext4:ext4_write_end [Tracepoint event] ext4:ext4_writepage [Tracepoint event] ext4:ext4_writepages [Tracepoint event] ext4:ext4_writepages_result [Tracepoint event] ext4:ext4_zero_range [Tracepoint event] fence:fence_annotate_wait_on [Tracepoint event] fence:fence_destroy [Tracepoint event] fence:fence_emit [Tracepoint event] fence:fence_enable_signal [Tracepoint event] fence:fence_init [Tracepoint event] fence:fence_signaled [Tracepoint event] fence:fence_wait_end [Tracepoint event] fence:fence_wait_start [Tracepoint event] filelock:break_lease_block [Tracepoint event] filelock:break_lease_noblock [Tracepoint event] filelock:break_lease_unblock [Tracepoint event] filelock:generic_add_lease [Tracepoint event] filelock:generic_delete_lease [Tracepoint event] filelock:time_out_leases [Tracepoint event] filemap:mm_filemap_add_to_page_cache [Tracepoint event] filemap:mm_filemap_delete_from_page_cache [Tracepoint event] ftrace:function [Tracepoint event] gpio:gpio_direction [Tracepoint event] gpio:gpio_value [Tracepoint event] hda:hda_get_response [Tracepoint event] hda:hda_send_cmd [Tracepoint event] hda:hda_unsol_event [Tracepoint event] hda:snd_hdac_stream_start [Tracepoint event] hda:snd_hdac_stream_stop [Tracepoint event] hda_controller:azx_get_position [Tracepoint event] hda_controller:azx_pcm_close [Tracepoint event] hda_controller:azx_pcm_hw_params [Tracepoint event] hda_controller:azx_pcm_open [Tracepoint event] hda_controller:azx_pcm_prepare [Tracepoint event] hda_controller:azx_pcm_trigger [Tracepoint event] hda_intel:azx_resume [Tracepoint event] hda_intel:azx_runtime_resume [Tracepoint event] hda_intel:azx_runtime_suspend [Tracepoint event] hda_intel:azx_suspend [Tracepoint event] i2c:i2c_read [Tracepoint event] i2c:i2c_reply [Tracepoint event] i2c:i2c_result [Tracepoint event] i2c:i2c_write [Tracepoint event] i2c:smbus_read [Tracepoint event] i2c:smbus_reply [Tracepoint event] i2c:smbus_result [Tracepoint event] i2c:smbus_write [Tracepoint event] iommu:add_device_to_group [Tracepoint event] iommu:attach_device_to_domain [Tracepoint event] iommu:detach_device_from_domain [Tracepoint event] iommu:io_page_fault [Tracepoint event] iommu:map [Tracepoint event] iommu:remove_device_from_group [Tracepoint event] iommu:unmap [Tracepoint event] irq:irq_handler_entry [Tracepoint event] irq:irq_handler_exit [Tracepoint event] irq:softirq_entry [Tracepoint event] irq:softirq_exit [Tracepoint event] irq:softirq_raise [Tracepoint event] irq_vectors:call_function_entry [Tracepoint event] irq_vectors:call_function_exit [Tracepoint event] irq_vectors:call_function_single_entry [Tracepoint event] irq_vectors:call_function_single_exit [Tracepoint event] irq_vectors:deferred_error_apic_entry [Tracepoint event] irq_vectors:deferred_error_apic_exit [Tracepoint event] irq_vectors:error_apic_entry [Tracepoint event] irq_vectors:error_apic_exit [Tracepoint event] irq_vectors:irq_work_entry [Tracepoint event] irq_vectors:irq_work_exit [Tracepoint event] irq_vectors:local_timer_entry [Tracepoint event] irq_vectors:local_timer_exit [Tracepoint event] irq_vectors:reschedule_entry [Tracepoint event] irq_vectors:reschedule_exit [Tracepoint event] irq_vectors:spurious_apic_entry [Tracepoint event] irq_vectors:spurious_apic_exit [Tracepoint event] irq_vectors:thermal_apic_entry [Tracepoint event] irq_vectors:thermal_apic_exit [Tracepoint event] irq_vectors:threshold_apic_entry [Tracepoint event] irq_vectors:threshold_apic_exit [Tracepoint event] irq_vectors:x86_platform_ipi_entry [Tracepoint event] irq_vectors:x86_platform_ipi_exit [Tracepoint event] jbd2:jbd2_checkpoint [Tracepoint event] jbd2:jbd2_checkpoint_stats [Tracepoint event] jbd2:jbd2_commit_flushing [Tracepoint event] jbd2:jbd2_commit_locking [Tracepoint event] jbd2:jbd2_commit_logging [Tracepoint event] jbd2:jbd2_drop_transaction [Tracepoint event] jbd2:jbd2_end_commit [Tracepoint event] jbd2:jbd2_handle_extend [Tracepoint event] jbd2:jbd2_handle_start [Tracepoint event] jbd2:jbd2_handle_stats [Tracepoint event] jbd2:jbd2_lock_buffer_stall [Tracepoint event] jbd2:jbd2_run_stats [Tracepoint event] jbd2:jbd2_start_commit [Tracepoint event] jbd2:jbd2_submit_inode_data [Tracepoint event] jbd2:jbd2_update_log_tail [Tracepoint event] jbd2:jbd2_write_superblock [Tracepoint event] kmem:kfree [Tracepoint event] kmem:kmalloc [Tracepoint event] kmem:kmalloc_node [Tracepoint event] kmem:kmem_cache_alloc [Tracepoint event] kmem:kmem_cache_alloc_node [Tracepoint event] kmem:kmem_cache_free [Tracepoint event] kmem:mm_page_alloc [Tracepoint event] kmem:mm_page_alloc_extfrag [Tracepoint event] kmem:mm_page_alloc_zone_locked [Tracepoint event] kmem:mm_page_free [Tracepoint event] kmem:mm_page_free_batched [Tracepoint event] kmem:mm_page_pcpu_drain [Tracepoint event] kvm:kvm_ack_irq [Tracepoint event] kvm:kvm_age_page [Tracepoint event] kvm:kvm_apic [Tracepoint event] kvm:kvm_apic_accept_irq [Tracepoint event] kvm:kvm_apic_ipi [Tracepoint event] kvm:kvm_async_pf_completed [Tracepoint event] kvm:kvm_async_pf_doublefault [Tracepoint event] kvm:kvm_async_pf_not_present [Tracepoint event] kvm:kvm_async_pf_ready [Tracepoint event] kvm:kvm_cpuid [Tracepoint event] kvm:kvm_cr [Tracepoint event] kvm:kvm_emulate_insn [Tracepoint event] kvm:kvm_enter_smm [Tracepoint event] kvm:kvm_entry [Tracepoint event] kvm:kvm_eoi [Tracepoint event] kvm:kvm_exit [Tracepoint event] kvm:kvm_fpu [Tracepoint event] kvm:kvm_hv_hypercall [Tracepoint event] kvm:kvm_hypercall [Tracepoint event] kvm:kvm_inj_exception [Tracepoint event] kvm:kvm_inj_virq [Tracepoint event] kvm:kvm_invlpga [Tracepoint event] kvm:kvm_ioapic_delayed_eoi_inj [Tracepoint event] kvm:kvm_ioapic_set_irq [Tracepoint event] kvm:kvm_mmio [Tracepoint event] kvm:kvm_msi_set_irq [Tracepoint event] kvm:kvm_msr [Tracepoint event] kvm:kvm_nested_intercepts [Tracepoint event] kvm:kvm_nested_intr_vmexit [Tracepoint event] kvm:kvm_nested_vmexit [Tracepoint event] kvm:kvm_nested_vmexit_inject [Tracepoint event] kvm:kvm_nested_vmrun [Tracepoint event] kvm:kvm_page_fault [Tracepoint event] kvm:kvm_pic_set_irq [Tracepoint event] kvm:kvm_pio [Tracepoint event] kvm:kvm_ple_window [Tracepoint event] kvm:kvm_pml_full [Tracepoint event] kvm:kvm_pv_eoi [Tracepoint event] kvm:kvm_pvclock_update [Tracepoint event] kvm:kvm_set_irq [Tracepoint event] kvm:kvm_skinit [Tracepoint event] kvm:kvm_track_tsc [Tracepoint event] kvm:kvm_try_async_get_page [Tracepoint event] kvm:kvm_update_master_clock [Tracepoint event] kvm:kvm_userspace_exit [Tracepoint event] kvm:kvm_vcpu_wakeup [Tracepoint event] kvm:kvm_wait_lapic_expire [Tracepoint event] kvm:kvm_write_tsc_offset [Tracepoint event] kvm:vcpu_match_mmio [Tracepoint event] kvmmmu:check_mmio_spte [Tracepoint event] kvmmmu:fast_page_fault [Tracepoint event] kvmmmu:handle_mmio_page_fault [Tracepoint event] kvmmmu:kvm_mmu_get_page [Tracepoint event] kvmmmu:kvm_mmu_invalidate_zap_all_pages [Tracepoint event] kvmmmu:kvm_mmu_pagetable_walk [Tracepoint event] kvmmmu:kvm_mmu_paging_element [Tracepoint event] kvmmmu:kvm_mmu_prepare_zap_page [Tracepoint event] kvmmmu:kvm_mmu_set_accessed_bit [Tracepoint event] kvmmmu:kvm_mmu_set_dirty_bit [Tracepoint event] kvmmmu:kvm_mmu_sync_page [Tracepoint event] kvmmmu:kvm_mmu_unsync_page [Tracepoint event] kvmmmu:kvm_mmu_walker_error [Tracepoint event] kvmmmu:mark_mmio_spte [Tracepoint event] libata:ata_eh_link_autopsy [Tracepoint event] libata:ata_eh_link_autopsy_qc [Tracepoint event] libata:ata_qc_complete_done [Tracepoint event] libata:ata_qc_complete_failed [Tracepoint event] libata:ata_qc_complete_internal [Tracepoint event] libata:ata_qc_issue [Tracepoint event] mce:mce_record [Tracepoint event] migrate:mm_migrate_pages [Tracepoint event] migrate:mm_numa_migrate_ratelimit [Tracepoint event] module:module_free [Tracepoint event] module:module_get [Tracepoint event] module:module_load [Tracepoint event] module:module_put [Tracepoint event] module:module_request [Tracepoint event] napi:napi_poll [Tracepoint event] net:napi_gro_frags_entry [Tracepoint event] net:napi_gro_receive_entry [Tracepoint event] net:net_dev_queue [Tracepoint event] net:net_dev_start_xmit [Tracepoint event] net:net_dev_xmit [Tracepoint event] net:netif_receive_skb [Tracepoint event] net:netif_receive_skb_entry [Tracepoint event] net:netif_rx [Tracepoint event] net:netif_rx_entry [Tracepoint event] net:netif_rx_ni_entry [Tracepoint event] nmi:nmi_handler [Tracepoint event] oom:oom_score_adj_update [Tracepoint event] pagemap:mm_lru_activate [Tracepoint event] pagemap:mm_lru_insertion [Tracepoint event] power:clock_disable [Tracepoint event] power:clock_enable [Tracepoint event] power:clock_set_rate [Tracepoint event] power:cpu_frequency [Tracepoint event] power:cpu_idle [Tracepoint event] power:dev_pm_qos_add_request [Tracepoint event] power:dev_pm_qos_remove_request [Tracepoint event] power:dev_pm_qos_update_request [Tracepoint event] power:device_pm_callback_end [Tracepoint event] power:device_pm_callback_start [Tracepoint event] power:pm_qos_add_request [Tracepoint event] power:pm_qos_remove_request [Tracepoint event] power:pm_qos_update_flags [Tracepoint event] power:pm_qos_update_request [Tracepoint event] power:pm_qos_update_request_timeout [Tracepoint event] power:pm_qos_update_target [Tracepoint event] power:power_domain_target [Tracepoint event] power:pstate_sample [Tracepoint event] power:suspend_resume [Tracepoint event] power:wakeup_source_activate [Tracepoint event] power:wakeup_source_deactivate [Tracepoint event] printk:console [Tracepoint event] radeon:radeon_bo_create [Tracepoint event] radeon:radeon_cs [Tracepoint event] radeon:radeon_fence_emit [Tracepoint event] radeon:radeon_fence_wait_begin [Tracepoint event] radeon:radeon_fence_wait_end [Tracepoint event] radeon:radeon_semaphore_signale [Tracepoint event] radeon:radeon_semaphore_wait [Tracepoint event] radeon:radeon_vm_bo_update [Tracepoint event] radeon:radeon_vm_flush [Tracepoint event] radeon:radeon_vm_grab_id [Tracepoint event] radeon:radeon_vm_set_page [Tracepoint event] random:add_device_randomness [Tracepoint event] random:add_disk_randomness [Tracepoint event] random:add_input_randomness [Tracepoint event] random:credit_entropy_bits [Tracepoint event] random:debit_entropy [Tracepoint event] random:extract_entropy [Tracepoint event] random:extract_entropy_user [Tracepoint event] random:get_random_bytes [Tracepoint event] random:get_random_bytes_arch [Tracepoint event] random:mix_pool_bytes [Tracepoint event] random:mix_pool_bytes_nolock [Tracepoint event] random:push_to_pool [Tracepoint event] random:random_read [Tracepoint event] random:urandom_read [Tracepoint event] random:xfer_secondary_pool [Tracepoint event] ras:aer_event [Tracepoint event] ras:extlog_mem_event [Tracepoint event] ras:mc_event [Tracepoint event] ras:memory_failure_event [Tracepoint event] raw_syscalls:sys_enter [Tracepoint event] raw_syscalls:sys_exit [Tracepoint event] rcu:rcu_utilization [Tracepoint event] regmap:regcache_drop_region [Tracepoint event] regmap:regcache_sync [Tracepoint event] regmap:regmap_async_complete_done [Tracepoint event] regmap:regmap_async_complete_start [Tracepoint event] regmap:regmap_async_io_complete [Tracepoint event] regmap:regmap_async_write_start [Tracepoint event] regmap:regmap_cache_bypass [Tracepoint event] regmap:regmap_cache_only [Tracepoint event] regmap:regmap_hw_read_done [Tracepoint event] regmap:regmap_hw_read_start [Tracepoint event] regmap:regmap_hw_write_done [Tracepoint event] regmap:regmap_hw_write_start [Tracepoint event] regmap:regmap_reg_read [Tracepoint event] regmap:regmap_reg_read_cache [Tracepoint event] regmap:regmap_reg_write [Tracepoint event] rpm:rpm_idle [Tracepoint event] rpm:rpm_resume [Tracepoint event] rpm:rpm_return_int [Tracepoint event] rpm:rpm_suspend [Tracepoint event] sched:sched_kthread_stop [Tracepoint event] sched:sched_kthread_stop_ret [Tracepoint event] sched:sched_migrate_task [Tracepoint event] sched:sched_move_numa [Tracepoint event] sched:sched_pi_setprio [Tracepoint event] sched:sched_process_exec [Tracepoint event] sched:sched_process_exit [Tracepoint event] sched:sched_process_fork [Tracepoint event] sched:sched_process_free [Tracepoint event] sched:sched_process_hang [Tracepoint event] sched:sched_process_wait [Tracepoint event] sched:sched_stat_blocked [Tracepoint event] sched:sched_stat_iowait [Tracepoint event] sched:sched_stat_runtime [Tracepoint event] sched:sched_stat_sleep [Tracepoint event] sched:sched_stat_wait [Tracepoint event] sched:sched_stick_numa [Tracepoint event] sched:sched_swap_numa [Tracepoint event] sched:sched_switch [Tracepoint event] sched:sched_wait_task [Tracepoint event] sched:sched_wake_idle_without_ipi [Tracepoint event] sched:sched_wakeup [Tracepoint event] sched:sched_wakeup_new [Tracepoint event] scsi:scsi_dispatch_cmd_done [Tracepoint event] scsi:scsi_dispatch_cmd_error [Tracepoint event] scsi:scsi_dispatch_cmd_start [Tracepoint event] scsi:scsi_dispatch_cmd_timeout [Tracepoint event] scsi:scsi_eh_wakeup [Tracepoint event] signal:signal_deliver [Tracepoint event] signal:signal_generate [Tracepoint event] skb:consume_skb [Tracepoint event] skb:kfree_skb [Tracepoint event] skb:skb_copy_datagram_iovec [Tracepoint event] sock:sock_exceed_buf_limit [Tracepoint event] sock:sock_rcvqueue_full [Tracepoint event] spi:spi_master_busy [Tracepoint event] spi:spi_master_idle [Tracepoint event] spi:spi_message_done [Tracepoint event] spi:spi_message_start [Tracepoint event] spi:spi_message_submit [Tracepoint event] spi:spi_transfer_start [Tracepoint event] spi:spi_transfer_stop [Tracepoint event] swiotlb:swiotlb_bounced [Tracepoint event] syscalls:sys_enter_accept [Tracepoint event] syscalls:sys_enter_accept4 [Tracepoint event] syscalls:sys_enter_access [Tracepoint event] syscalls:sys_enter_acct [Tracepoint event] syscalls:sys_enter_add_key [Tracepoint event] syscalls:sys_enter_adjtimex [Tracepoint event] syscalls:sys_enter_alarm [Tracepoint event] syscalls:sys_enter_bind [Tracepoint event] syscalls:sys_enter_brk [Tracepoint event] syscalls:sys_enter_capget [Tracepoint event] syscalls:sys_enter_capset [Tracepoint event] syscalls:sys_enter_chdir [Tracepoint event] syscalls:sys_enter_chmod [Tracepoint event] syscalls:sys_enter_chown [Tracepoint event] syscalls:sys_enter_chroot [Tracepoint event] syscalls:sys_enter_clock_adjtime [Tracepoint event] syscalls:sys_enter_clock_getres [Tracepoint event] syscalls:sys_enter_clock_gettime [Tracepoint event] syscalls:sys_enter_clock_nanosleep [Tracepoint event] syscalls:sys_enter_clock_settime [Tracepoint event] syscalls:sys_enter_close [Tracepoint event] syscalls:sys_enter_connect [Tracepoint event] syscalls:sys_enter_creat [Tracepoint event] syscalls:sys_enter_delete_module [Tracepoint event] syscalls:sys_enter_dup [Tracepoint event] syscalls:sys_enter_dup2 [Tracepoint event] syscalls:sys_enter_dup3 [Tracepoint event] syscalls:sys_enter_epoll_create [Tracepoint event] syscalls:sys_enter_epoll_create1 [Tracepoint event] syscalls:sys_enter_epoll_ctl [Tracepoint event] syscalls:sys_enter_epoll_pwait [Tracepoint event] syscalls:sys_enter_epoll_wait [Tracepoint event] syscalls:sys_enter_eventfd [Tracepoint event] syscalls:sys_enter_eventfd2 [Tracepoint event] syscalls:sys_enter_exit [Tracepoint event] syscalls:sys_enter_exit_group [Tracepoint event] syscalls:sys_enter_faccessat [Tracepoint event] syscalls:sys_enter_fadvise64 [Tracepoint event] syscalls:sys_enter_fallocate [Tracepoint event] syscalls:sys_enter_fanotify_init [Tracepoint event] syscalls:sys_enter_fanotify_mark [Tracepoint event] syscalls:sys_enter_fchdir [Tracepoint event] syscalls:sys_enter_fchmod [Tracepoint event] syscalls:sys_enter_fchmodat [Tracepoint event] syscalls:sys_enter_fchown [Tracepoint event] syscalls:sys_enter_fchownat [Tracepoint event] syscalls:sys_enter_fcntl [Tracepoint event] syscalls:sys_enter_fdatasync [Tracepoint event] syscalls:sys_enter_fgetxattr [Tracepoint event] syscalls:sys_enter_finit_module [Tracepoint event] syscalls:sys_enter_flistxattr [Tracepoint event] syscalls:sys_enter_flock [Tracepoint event] syscalls:sys_enter_fremovexattr [Tracepoint event] syscalls:sys_enter_fsetxattr [Tracepoint event] syscalls:sys_enter_fstatfs [Tracepoint event] syscalls:sys_enter_fsync [Tracepoint event] syscalls:sys_enter_ftruncate [Tracepoint event] syscalls:sys_enter_futex [Tracepoint event] syscalls:sys_enter_futimesat [Tracepoint event] syscalls:sys_enter_get_mempolicy [Tracepoint event] syscalls:sys_enter_get_robust_list [Tracepoint event] syscalls:sys_enter_getcpu [Tracepoint event] syscalls:sys_enter_getcwd [Tracepoint event] syscalls:sys_enter_getdents [Tracepoint event] syscalls:sys_enter_getdents64 [Tracepoint event] syscalls:sys_enter_getegid [Tracepoint event] syscalls:sys_enter_geteuid [Tracepoint event] syscalls:sys_enter_getgid [Tracepoint event] syscalls:sys_enter_getgroups [Tracepoint event] syscalls:sys_enter_getitimer [Tracepoint event] syscalls:sys_enter_getpeername [Tracepoint event] syscalls:sys_enter_getpgid [Tracepoint event] syscalls:sys_enter_getpgrp [Tracepoint event] syscalls:sys_enter_getpid [Tracepoint event] syscalls:sys_enter_getppid [Tracepoint event] syscalls:sys_enter_getpriority [Tracepoint event] syscalls:sys_enter_getrandom [Tracepoint event] syscalls:sys_enter_getresgid [Tracepoint event] syscalls:sys_enter_getresuid [Tracepoint event] syscalls:sys_enter_getrlimit [Tracepoint event] syscalls:sys_enter_getrusage [Tracepoint event] syscalls:sys_enter_getsid [Tracepoint event] syscalls:sys_enter_getsockname [Tracepoint event] syscalls:sys_enter_getsockopt [Tracepoint event] syscalls:sys_enter_gettid [Tracepoint event] syscalls:sys_enter_gettimeofday [Tracepoint event] syscalls:sys_enter_getuid [Tracepoint event] syscalls:sys_enter_getxattr [Tracepoint event] syscalls:sys_enter_init_module [Tracepoint event] syscalls:sys_enter_inotify_add_watch [Tracepoint event] syscalls:sys_enter_inotify_init [Tracepoint event] syscalls:sys_enter_inotify_init1 [Tracepoint event] syscalls:sys_enter_inotify_rm_watch [Tracepoint event] syscalls:sys_enter_io_cancel [Tracepoint event] syscalls:sys_enter_io_destroy [Tracepoint event] syscalls:sys_enter_io_getevents [Tracepoint event] syscalls:sys_enter_io_setup [Tracepoint event] syscalls:sys_enter_io_submit [Tracepoint event] syscalls:sys_enter_ioctl [Tracepoint event] syscalls:sys_enter_iopl [Tracepoint event] syscalls:sys_enter_ioprio_get [Tracepoint event] syscalls:sys_enter_ioprio_set [Tracepoint event] syscalls:sys_enter_kcmp [Tracepoint event] syscalls:sys_enter_kexec_load [Tracepoint event] syscalls:sys_enter_keyctl [Tracepoint event] syscalls:sys_enter_kill [Tracepoint event] syscalls:sys_enter_lchown [Tracepoint event] syscalls:sys_enter_lgetxattr [Tracepoint event] syscalls:sys_enter_link [Tracepoint event] syscalls:sys_enter_linkat [Tracepoint event] syscalls:sys_enter_listen [Tracepoint event] syscalls:sys_enter_listxattr [Tracepoint event] syscalls:sys_enter_llistxattr [Tracepoint event] syscalls:sys_enter_lookup_dcookie [Tracepoint event] syscalls:sys_enter_lremovexattr [Tracepoint event] syscalls:sys_enter_lseek [Tracepoint event] syscalls:sys_enter_lsetxattr [Tracepoint event] syscalls:sys_enter_madvise [Tracepoint event] syscalls:sys_enter_mbind [Tracepoint event] syscalls:sys_enter_memfd_create [Tracepoint event] syscalls:sys_enter_migrate_pages [Tracepoint event] syscalls:sys_enter_mincore [Tracepoint event] syscalls:sys_enter_mkdir [Tracepoint event] syscalls:sys_enter_mkdirat [Tracepoint event] syscalls:sys_enter_mknod [Tracepoint event] syscalls:sys_enter_mknodat [Tracepoint event] syscalls:sys_enter_mlock [Tracepoint event] syscalls:sys_enter_mlockall [Tracepoint event] syscalls:sys_enter_mmap [Tracepoint event] syscalls:sys_enter_mount [Tracepoint event] syscalls:sys_enter_move_pages [Tracepoint event] syscalls:sys_enter_mprotect [Tracepoint event] syscalls:sys_enter_mq_getsetattr [Tracepoint event] syscalls:sys_enter_mq_notify [Tracepoint event] syscalls:sys_enter_mq_open [Tracepoint event] syscalls:sys_enter_mq_timedreceive [Tracepoint event] syscalls:sys_enter_mq_timedsend [Tracepoint event] syscalls:sys_enter_mq_unlink [Tracepoint event] syscalls:sys_enter_mremap [Tracepoint event] syscalls:sys_enter_msgctl [Tracepoint event] syscalls:sys_enter_msgget [Tracepoint event] syscalls:sys_enter_msgrcv [Tracepoint event] syscalls:sys_enter_msgsnd [Tracepoint event] syscalls:sys_enter_msync [Tracepoint event] syscalls:sys_enter_munlock [Tracepoint event] syscalls:sys_enter_munlockall [Tracepoint event] syscalls:sys_enter_munmap [Tracepoint event] syscalls:sys_enter_name_to_handle_at [Tracepoint event] syscalls:sys_enter_nanosleep [Tracepoint event] syscalls:sys_enter_newfstat [Tracepoint event] syscalls:sys_enter_newfstatat [Tracepoint event] syscalls:sys_enter_newlstat [Tracepoint event] syscalls:sys_enter_newstat [Tracepoint event] syscalls:sys_enter_newuname [Tracepoint event] syscalls:sys_enter_open [Tracepoint event] syscalls:sys_enter_open_by_handle_at [Tracepoint event] syscalls:sys_enter_openat [Tracepoint event] syscalls:sys_enter_pause [Tracepoint event] syscalls:sys_enter_perf_event_open [Tracepoint event] syscalls:sys_enter_personality [Tracepoint event] syscalls:sys_enter_pipe [Tracepoint event] syscalls:sys_enter_pipe2 [Tracepoint event] syscalls:sys_enter_pivot_root [Tracepoint event] syscalls:sys_enter_poll [Tracepoint event] syscalls:sys_enter_ppoll [Tracepoint event] syscalls:sys_enter_prctl [Tracepoint event] syscalls:sys_enter_pread64 [Tracepoint event] syscalls:sys_enter_preadv [Tracepoint event] syscalls:sys_enter_prlimit64 [Tracepoint event] syscalls:sys_enter_process_vm_readv [Tracepoint event] syscalls:sys_enter_process_vm_writev [Tracepoint event] syscalls:sys_enter_pselect6 [Tracepoint event] syscalls:sys_enter_ptrace [Tracepoint event] syscalls:sys_enter_pwrite64 [Tracepoint event] syscalls:sys_enter_pwritev [Tracepoint event] syscalls:sys_enter_quotactl [Tracepoint event] syscalls:sys_enter_read [Tracepoint event] syscalls:sys_enter_readahead [Tracepoint event] syscalls:sys_enter_readlink [Tracepoint event] syscalls:sys_enter_readlinkat [Tracepoint event] syscalls:sys_enter_readv [Tracepoint event] syscalls:sys_enter_reboot [Tracepoint event] syscalls:sys_enter_recvfrom [Tracepoint event] syscalls:sys_enter_recvmmsg [Tracepoint event] syscalls:sys_enter_recvmsg [Tracepoint event] syscalls:sys_enter_remap_file_pages [Tracepoint event] syscalls:sys_enter_removexattr [Tracepoint event] syscalls:sys_enter_rename [Tracepoint event] syscalls:sys_enter_renameat [Tracepoint event] syscalls:sys_enter_renameat2 [Tracepoint event] syscalls:sys_enter_request_key [Tracepoint event] syscalls:sys_enter_restart_syscall [Tracepoint event] syscalls:sys_enter_rmdir [Tracepoint event] syscalls:sys_enter_rt_sigaction [Tracepoint event] syscalls:sys_enter_rt_sigpending [Tracepoint event] syscalls:sys_enter_rt_sigprocmask [Tracepoint event] syscalls:sys_enter_rt_sigqueueinfo [Tracepoint event] syscalls:sys_enter_rt_sigsuspend [Tracepoint event] syscalls:sys_enter_rt_sigtimedwait [Tracepoint event] syscalls:sys_enter_rt_tgsigqueueinfo [Tracepoint event] syscalls:sys_enter_sched_get_priority_max [Tracepoint event] syscalls:sys_enter_sched_get_priority_min [Tracepoint event] syscalls:sys_enter_sched_getaffinity [Tracepoint event] syscalls:sys_enter_sched_getattr [Tracepoint event] syscalls:sys_enter_sched_getparam [Tracepoint event] syscalls:sys_enter_sched_getscheduler [Tracepoint event] syscalls:sys_enter_sched_rr_get_interval [Tracepoint event] syscalls:sys_enter_sched_setaffinity [Tracepoint event] syscalls:sys_enter_sched_setattr [Tracepoint event] syscalls:sys_enter_sched_setparam [Tracepoint event] syscalls:sys_enter_sched_setscheduler [Tracepoint event] syscalls:sys_enter_sched_yield [Tracepoint event] syscalls:sys_enter_seccomp [Tracepoint event] syscalls:sys_enter_select [Tracepoint event] syscalls:sys_enter_semctl [Tracepoint event] syscalls:sys_enter_semget [Tracepoint event] syscalls:sys_enter_semop [Tracepoint event] syscalls:sys_enter_semtimedop [Tracepoint event] syscalls:sys_enter_sendfile64 [Tracepoint event] syscalls:sys_enter_sendmmsg [Tracepoint event] syscalls:sys_enter_sendmsg [Tracepoint event] syscalls:sys_enter_sendto [Tracepoint event] syscalls:sys_enter_set_mempolicy [Tracepoint event] syscalls:sys_enter_set_robust_list [Tracepoint event] syscalls:sys_enter_set_tid_address [Tracepoint event] syscalls:sys_enter_setdomainname [Tracepoint event] syscalls:sys_enter_setfsgid [Tracepoint event] syscalls:sys_enter_setfsuid [Tracepoint event] syscalls:sys_enter_setgid [Tracepoint event] syscalls:sys_enter_setgroups [Tracepoint event] syscalls:sys_enter_sethostname [Tracepoint event] syscalls:sys_enter_setitimer [Tracepoint event] syscalls:sys_enter_setns [Tracepoint event] syscalls:sys_enter_setpgid [Tracepoint event] syscalls:sys_enter_setpriority [Tracepoint event] syscalls:sys_enter_setregid [Tracepoint event] syscalls:sys_enter_setresgid [Tracepoint event] syscalls:sys_enter_setresuid [Tracepoint event] syscalls:sys_enter_setreuid [Tracepoint event] syscalls:sys_enter_setrlimit [Tracepoint event] syscalls:sys_enter_setsid [Tracepoint event] syscalls:sys_enter_setsockopt [Tracepoint event] syscalls:sys_enter_settimeofday [Tracepoint event] syscalls:sys_enter_setuid [Tracepoint event] syscalls:sys_enter_setxattr [Tracepoint event] syscalls:sys_enter_shmat [Tracepoint event] syscalls:sys_enter_shmctl [Tracepoint event] syscalls:sys_enter_shmdt [Tracepoint event] syscalls:sys_enter_shmget [Tracepoint event] syscalls:sys_enter_shutdown [Tracepoint event] syscalls:sys_enter_sigaltstack [Tracepoint event] syscalls:sys_enter_signalfd [Tracepoint event] syscalls:sys_enter_signalfd4 [Tracepoint event] syscalls:sys_enter_socket [Tracepoint event] syscalls:sys_enter_socketpair [Tracepoint event] syscalls:sys_enter_splice [Tracepoint event] syscalls:sys_enter_statfs [Tracepoint event] syscalls:sys_enter_swapoff [Tracepoint event] syscalls:sys_enter_swapon [Tracepoint event] syscalls:sys_enter_symlink [Tracepoint event] syscalls:sys_enter_symlinkat [Tracepoint event] syscalls:sys_enter_sync [Tracepoint event] syscalls:sys_enter_sync_file_range [Tracepoint event] syscalls:sys_enter_syncfs [Tracepoint event] syscalls:sys_enter_sysctl [Tracepoint event] syscalls:sys_enter_sysfs [Tracepoint event] syscalls:sys_enter_sysinfo [Tracepoint event] syscalls:sys_enter_syslog [Tracepoint event] syscalls:sys_enter_tee [Tracepoint event] syscalls:sys_enter_tgkill [Tracepoint event] syscalls:sys_enter_time [Tracepoint event] syscalls:sys_enter_timer_create [Tracepoint event] syscalls:sys_enter_timer_delete [Tracepoint event] syscalls:sys_enter_timer_getoverrun [Tracepoint event] syscalls:sys_enter_timer_gettime [Tracepoint event] syscalls:sys_enter_timer_settime [Tracepoint event] syscalls:sys_enter_timerfd_create [Tracepoint event] syscalls:sys_enter_timerfd_gettime [Tracepoint event] syscalls:sys_enter_timerfd_settime [Tracepoint event] syscalls:sys_enter_times [Tracepoint event] syscalls:sys_enter_tkill [Tracepoint event] syscalls:sys_enter_truncate [Tracepoint event] syscalls:sys_enter_umask [Tracepoint event] syscalls:sys_enter_umount [Tracepoint event] syscalls:sys_enter_unlink [Tracepoint event] syscalls:sys_enter_unlinkat [Tracepoint event] syscalls:sys_enter_unshare [Tracepoint event] syscalls:sys_enter_ustat [Tracepoint event] syscalls:sys_enter_utime [Tracepoint event] syscalls:sys_enter_utimensat [Tracepoint event] syscalls:sys_enter_utimes [Tracepoint event] syscalls:sys_enter_vhangup [Tracepoint event] syscalls:sys_enter_vmsplice [Tracepoint event] syscalls:sys_enter_wait4 [Tracepoint event] syscalls:sys_enter_waitid [Tracepoint event] syscalls:sys_enter_write [Tracepoint event] syscalls:sys_enter_writev [Tracepoint event] syscalls:sys_exit_accept [Tracepoint event] syscalls:sys_exit_accept4 [Tracepoint event] syscalls:sys_exit_access [Tracepoint event] syscalls:sys_exit_acct [Tracepoint event] syscalls:sys_exit_add_key [Tracepoint event] syscalls:sys_exit_adjtimex [Tracepoint event] syscalls:sys_exit_alarm [Tracepoint event] syscalls:sys_exit_bind [Tracepoint event] syscalls:sys_exit_brk [Tracepoint event] syscalls:sys_exit_capget [Tracepoint event] syscalls:sys_exit_capset [Tracepoint event] syscalls:sys_exit_chdir [Tracepoint event] syscalls:sys_exit_chmod [Tracepoint event] syscalls:sys_exit_chown [Tracepoint event] syscalls:sys_exit_chroot [Tracepoint event] syscalls:sys_exit_clock_adjtime [Tracepoint event] syscalls:sys_exit_clock_getres [Tracepoint event] syscalls:sys_exit_clock_gettime [Tracepoint event] syscalls:sys_exit_clock_nanosleep [Tracepoint event] syscalls:sys_exit_clock_settime [Tracepoint event] syscalls:sys_exit_close [Tracepoint event] syscalls:sys_exit_connect [Tracepoint event] syscalls:sys_exit_creat [Tracepoint event] syscalls:sys_exit_delete_module [Tracepoint event] syscalls:sys_exit_dup [Tracepoint event] syscalls:sys_exit_dup2 [Tracepoint event] syscalls:sys_exit_dup3 [Tracepoint event] syscalls:sys_exit_epoll_create [Tracepoint event] syscalls:sys_exit_epoll_create1 [Tracepoint event] syscalls:sys_exit_epoll_ctl [Tracepoint event] syscalls:sys_exit_epoll_pwait [Tracepoint event] syscalls:sys_exit_epoll_wait [Tracepoint event] syscalls:sys_exit_eventfd [Tracepoint event] syscalls:sys_exit_eventfd2 [Tracepoint event] syscalls:sys_exit_exit [Tracepoint event] syscalls:sys_exit_exit_group [Tracepoint event] syscalls:sys_exit_faccessat [Tracepoint event] syscalls:sys_exit_fadvise64 [Tracepoint event] syscalls:sys_exit_fallocate [Tracepoint event] syscalls:sys_exit_fanotify_init [Tracepoint event] syscalls:sys_exit_fanotify_mark [Tracepoint event] syscalls:sys_exit_fchdir [Tracepoint event] syscalls:sys_exit_fchmod [Tracepoint event] syscalls:sys_exit_fchmodat [Tracepoint event] syscalls:sys_exit_fchown [Tracepoint event] syscalls:sys_exit_fchownat [Tracepoint event] syscalls:sys_exit_fcntl [Tracepoint event] syscalls:sys_exit_fdatasync [Tracepoint event] syscalls:sys_exit_fgetxattr [Tracepoint event] syscalls:sys_exit_finit_module [Tracepoint event] syscalls:sys_exit_flistxattr [Tracepoint event] syscalls:sys_exit_flock [Tracepoint event] syscalls:sys_exit_fremovexattr [Tracepoint event] syscalls:sys_exit_fsetxattr [Tracepoint event] syscalls:sys_exit_fstatfs [Tracepoint event] syscalls:sys_exit_fsync [Tracepoint event] syscalls:sys_exit_ftruncate [Tracepoint event] syscalls:sys_exit_futex [Tracepoint event] syscalls:sys_exit_futimesat [Tracepoint event] syscalls:sys_exit_get_mempolicy [Tracepoint event] syscalls:sys_exit_get_robust_list [Tracepoint event] syscalls:sys_exit_getcpu [Tracepoint event] syscalls:sys_exit_getcwd [Tracepoint event] syscalls:sys_exit_getdents [Tracepoint event] syscalls:sys_exit_getdents64 [Tracepoint event] syscalls:sys_exit_getegid [Tracepoint event] syscalls:sys_exit_geteuid [Tracepoint event] syscalls:sys_exit_getgid [Tracepoint event] syscalls:sys_exit_getgroups [Tracepoint event] syscalls:sys_exit_getitimer [Tracepoint event] syscalls:sys_exit_getpeername [Tracepoint event] syscalls:sys_exit_getpgid [Tracepoint event] syscalls:sys_exit_getpgrp [Tracepoint event] syscalls:sys_exit_getpid [Tracepoint event] syscalls:sys_exit_getppid [Tracepoint event] syscalls:sys_exit_getpriority [Tracepoint event] syscalls:sys_exit_getrandom [Tracepoint event] syscalls:sys_exit_getresgid [Tracepoint event] syscalls:sys_exit_getresuid [Tracepoint event] syscalls:sys_exit_getrlimit [Tracepoint event] syscalls:sys_exit_getrusage [Tracepoint event] syscalls:sys_exit_getsid [Tracepoint event] syscalls:sys_exit_getsockname [Tracepoint event] syscalls:sys_exit_getsockopt [Tracepoint event] syscalls:sys_exit_gettid [Tracepoint event] syscalls:sys_exit_gettimeofday [Tracepoint event] syscalls:sys_exit_getuid [Tracepoint event] syscalls:sys_exit_getxattr [Tracepoint event] syscalls:sys_exit_init_module [Tracepoint event] syscalls:sys_exit_inotify_add_watch [Tracepoint event] syscalls:sys_exit_inotify_init [Tracepoint event] syscalls:sys_exit_inotify_init1 [Tracepoint event] syscalls:sys_exit_inotify_rm_watch [Tracepoint event] syscalls:sys_exit_io_cancel [Tracepoint event] syscalls:sys_exit_io_destroy [Tracepoint event] syscalls:sys_exit_io_getevents [Tracepoint event] syscalls:sys_exit_io_setup [Tracepoint event] syscalls:sys_exit_io_submit [Tracepoint event] syscalls:sys_exit_ioctl [Tracepoint event] syscalls:sys_exit_iopl [Tracepoint event] syscalls:sys_exit_ioprio_get [Tracepoint event] syscalls:sys_exit_ioprio_set [Tracepoint event] syscalls:sys_exit_kcmp [Tracepoint event] syscalls:sys_exit_kexec_load [Tracepoint event] syscalls:sys_exit_keyctl [Tracepoint event] syscalls:sys_exit_kill [Tracepoint event] syscalls:sys_exit_lchown [Tracepoint event] syscalls:sys_exit_lgetxattr [Tracepoint event] syscalls:sys_exit_link [Tracepoint event] syscalls:sys_exit_linkat [Tracepoint event] syscalls:sys_exit_listen [Tracepoint event] syscalls:sys_exit_listxattr [Tracepoint event] syscalls:sys_exit_llistxattr [Tracepoint event] syscalls:sys_exit_lookup_dcookie [Tracepoint event] syscalls:sys_exit_lremovexattr [Tracepoint event] syscalls:sys_exit_lseek [Tracepoint event] syscalls:sys_exit_lsetxattr [Tracepoint event] syscalls:sys_exit_madvise [Tracepoint event] syscalls:sys_exit_mbind [Tracepoint event] syscalls:sys_exit_memfd_create [Tracepoint event] syscalls:sys_exit_migrate_pages [Tracepoint event] syscalls:sys_exit_mincore [Tracepoint event] syscalls:sys_exit_mkdir [Tracepoint event] syscalls:sys_exit_mkdirat [Tracepoint event] syscalls:sys_exit_mknod [Tracepoint event] syscalls:sys_exit_mknodat [Tracepoint event] syscalls:sys_exit_mlock [Tracepoint event] syscalls:sys_exit_mlockall [Tracepoint event] syscalls:sys_exit_mmap [Tracepoint event] syscalls:sys_exit_mount [Tracepoint event] syscalls:sys_exit_move_pages [Tracepoint event] syscalls:sys_exit_mprotect [Tracepoint event] syscalls:sys_exit_mq_getsetattr [Tracepoint event] syscalls:sys_exit_mq_notify [Tracepoint event] syscalls:sys_exit_mq_open [Tracepoint event] syscalls:sys_exit_mq_timedreceive [Tracepoint event] syscalls:sys_exit_mq_timedsend [Tracepoint event] syscalls:sys_exit_mq_unlink [Tracepoint event] syscalls:sys_exit_mremap [Tracepoint event] syscalls:sys_exit_msgctl [Tracepoint event] syscalls:sys_exit_msgget [Tracepoint event] syscalls:sys_exit_msgrcv [Tracepoint event] syscalls:sys_exit_msgsnd [Tracepoint event] syscalls:sys_exit_msync [Tracepoint event] syscalls:sys_exit_munlock [Tracepoint event] syscalls:sys_exit_munlockall [Tracepoint event] syscalls:sys_exit_munmap [Tracepoint event] syscalls:sys_exit_name_to_handle_at [Tracepoint event] syscalls:sys_exit_nanosleep [Tracepoint event] syscalls:sys_exit_newfstat [Tracepoint event] syscalls:sys_exit_newfstatat [Tracepoint event] syscalls:sys_exit_newlstat [Tracepoint event] syscalls:sys_exit_newstat [Tracepoint event] syscalls:sys_exit_newuname [Tracepoint event] syscalls:sys_exit_open [Tracepoint event] syscalls:sys_exit_open_by_handle_at [Tracepoint event] syscalls:sys_exit_openat [Tracepoint event] syscalls:sys_exit_pause [Tracepoint event] syscalls:sys_exit_perf_event_open [Tracepoint event] syscalls:sys_exit_personality [Tracepoint event] syscalls:sys_exit_pipe [Tracepoint event] syscalls:sys_exit_pipe2 [Tracepoint event] syscalls:sys_exit_pivot_root [Tracepoint event] syscalls:sys_exit_poll [Tracepoint event] syscalls:sys_exit_ppoll [Tracepoint event] syscalls:sys_exit_prctl [Tracepoint event] syscalls:sys_exit_pread64 [Tracepoint event] syscalls:sys_exit_preadv [Tracepoint event] syscalls:sys_exit_prlimit64 [Tracepoint event] syscalls:sys_exit_process_vm_readv [Tracepoint event] syscalls:sys_exit_process_vm_writev [Tracepoint event] syscalls:sys_exit_pselect6 [Tracepoint event] syscalls:sys_exit_ptrace [Tracepoint event] syscalls:sys_exit_pwrite64 [Tracepoint event] syscalls:sys_exit_pwritev [Tracepoint event] syscalls:sys_exit_quotactl [Tracepoint event] syscalls:sys_exit_read [Tracepoint event] syscalls:sys_exit_readahead [Tracepoint event] syscalls:sys_exit_readlink [Tracepoint event] syscalls:sys_exit_readlinkat [Tracepoint event] syscalls:sys_exit_readv [Tracepoint event] syscalls:sys_exit_reboot [Tracepoint event] syscalls:sys_exit_recvfrom [Tracepoint event] syscalls:sys_exit_recvmmsg [Tracepoint event] syscalls:sys_exit_recvmsg [Tracepoint event] syscalls:sys_exit_remap_file_pages [Tracepoint event] syscalls:sys_exit_removexattr [Tracepoint event] syscalls:sys_exit_rename [Tracepoint event] syscalls:sys_exit_renameat [Tracepoint event] syscalls:sys_exit_renameat2 [Tracepoint event] syscalls:sys_exit_request_key [Tracepoint event] syscalls:sys_exit_restart_syscall [Tracepoint event] syscalls:sys_exit_rmdir [Tracepoint event] syscalls:sys_exit_rt_sigaction [Tracepoint event] syscalls:sys_exit_rt_sigpending [Tracepoint event] syscalls:sys_exit_rt_sigprocmask [Tracepoint event] syscalls:sys_exit_rt_sigqueueinfo [Tracepoint event] syscalls:sys_exit_rt_sigsuspend [Tracepoint event] syscalls:sys_exit_rt_sigtimedwait [Tracepoint event] syscalls:sys_exit_rt_tgsigqueueinfo [Tracepoint event] syscalls:sys_exit_sched_get_priority_max [Tracepoint event] syscalls:sys_exit_sched_get_priority_min [Tracepoint event] syscalls:sys_exit_sched_getaffinity [Tracepoint event] syscalls:sys_exit_sched_getattr [Tracepoint event] syscalls:sys_exit_sched_getparam [Tracepoint event] syscalls:sys_exit_sched_getscheduler [Tracepoint event] syscalls:sys_exit_sched_rr_get_interval [Tracepoint event] syscalls:sys_exit_sched_setaffinity [Tracepoint event] syscalls:sys_exit_sched_setattr [Tracepoint event] syscalls:sys_exit_sched_setparam [Tracepoint event] syscalls:sys_exit_sched_setscheduler [Tracepoint event] syscalls:sys_exit_sched_yield [Tracepoint event] syscalls:sys_exit_seccomp [Tracepoint event] syscalls:sys_exit_select [Tracepoint event] syscalls:sys_exit_semctl [Tracepoint event] syscalls:sys_exit_semget [Tracepoint event] syscalls:sys_exit_semop [Tracepoint event] syscalls:sys_exit_semtimedop [Tracepoint event] syscalls:sys_exit_sendfile64 [Tracepoint event] syscalls:sys_exit_sendmmsg [Tracepoint event] syscalls:sys_exit_sendmsg [Tracepoint event] syscalls:sys_exit_sendto [Tracepoint event] syscalls:sys_exit_set_mempolicy [Tracepoint event] syscalls:sys_exit_set_robust_list [Tracepoint event] syscalls:sys_exit_set_tid_address [Tracepoint event] syscalls:sys_exit_setdomainname [Tracepoint event] syscalls:sys_exit_setfsgid [Tracepoint event] syscalls:sys_exit_setfsuid [Tracepoint event] syscalls:sys_exit_setgid [Tracepoint event] syscalls:sys_exit_setgroups [Tracepoint event] syscalls:sys_exit_sethostname [Tracepoint event] syscalls:sys_exit_setitimer [Tracepoint event] syscalls:sys_exit_setns [Tracepoint event] syscalls:sys_exit_setpgid [Tracepoint event] syscalls:sys_exit_setpriority [Tracepoint event] syscalls:sys_exit_setregid [Tracepoint event] syscalls:sys_exit_setresgid [Tracepoint event] syscalls:sys_exit_setresuid [Tracepoint event] syscalls:sys_exit_setreuid [Tracepoint event] syscalls:sys_exit_setrlimit [Tracepoint event] syscalls:sys_exit_setsid [Tracepoint event] syscalls:sys_exit_setsockopt [Tracepoint event] syscalls:sys_exit_settimeofday [Tracepoint event] syscalls:sys_exit_setuid [Tracepoint event] syscalls:sys_exit_setxattr [Tracepoint event] syscalls:sys_exit_shmat [Tracepoint event] syscalls:sys_exit_shmctl [Tracepoint event] syscalls:sys_exit_shmdt [Tracepoint event] syscalls:sys_exit_shmget [Tracepoint event] syscalls:sys_exit_shutdown [Tracepoint event] syscalls:sys_exit_sigaltstack [Tracepoint event] syscalls:sys_exit_signalfd [Tracepoint event] syscalls:sys_exit_signalfd4 [Tracepoint event] syscalls:sys_exit_socket [Tracepoint event] syscalls:sys_exit_socketpair [Tracepoint event] syscalls:sys_exit_splice [Tracepoint event] syscalls:sys_exit_statfs [Tracepoint event] syscalls:sys_exit_swapoff [Tracepoint event] syscalls:sys_exit_swapon [Tracepoint event] syscalls:sys_exit_symlink [Tracepoint event] syscalls:sys_exit_symlinkat [Tracepoint event] syscalls:sys_exit_sync [Tracepoint event] syscalls:sys_exit_sync_file_range [Tracepoint event] syscalls:sys_exit_syncfs [Tracepoint event] syscalls:sys_exit_sysctl [Tracepoint event] syscalls:sys_exit_sysfs [Tracepoint event] syscalls:sys_exit_sysinfo [Tracepoint event] syscalls:sys_exit_syslog [Tracepoint event] syscalls:sys_exit_tee [Tracepoint event] syscalls:sys_exit_tgkill [Tracepoint event] syscalls:sys_exit_time [Tracepoint event] syscalls:sys_exit_timer_create [Tracepoint event] syscalls:sys_exit_timer_delete [Tracepoint event] syscalls:sys_exit_timer_getoverrun [Tracepoint event] syscalls:sys_exit_timer_gettime [Tracepoint event] syscalls:sys_exit_timer_settime [Tracepoint event] syscalls:sys_exit_timerfd_create [Tracepoint event] syscalls:sys_exit_timerfd_gettime [Tracepoint event] syscalls:sys_exit_timerfd_settime [Tracepoint event] syscalls:sys_exit_times [Tracepoint event] syscalls:sys_exit_tkill [Tracepoint event] syscalls:sys_exit_truncate [Tracepoint event] syscalls:sys_exit_umask [Tracepoint event] syscalls:sys_exit_umount [Tracepoint event] syscalls:sys_exit_unlink [Tracepoint event] syscalls:sys_exit_unlinkat [Tracepoint event] syscalls:sys_exit_unshare [Tracepoint event] syscalls:sys_exit_ustat [Tracepoint event] syscalls:sys_exit_utime [Tracepoint event] syscalls:sys_exit_utimensat [Tracepoint event] syscalls:sys_exit_utimes [Tracepoint event] syscalls:sys_exit_vhangup [Tracepoint event] syscalls:sys_exit_vmsplice [Tracepoint event] syscalls:sys_exit_wait4 [Tracepoint event] syscalls:sys_exit_waitid [Tracepoint event] syscalls:sys_exit_write [Tracepoint event] syscalls:sys_exit_writev [Tracepoint event] task:task_newtask [Tracepoint event] task:task_rename [Tracepoint event] thermal:cdev_update [Tracepoint event] thermal:thermal_power_cpu_get_power [Tracepoint event] thermal:thermal_power_cpu_limit [Tracepoint event] thermal:thermal_temperature [Tracepoint event] thermal:thermal_zone_trip [Tracepoint event] timer:hrtimer_cancel [Tracepoint event] timer:hrtimer_expire_entry [Tracepoint event] timer:hrtimer_expire_exit [Tracepoint event] timer:hrtimer_init [Tracepoint event] timer:hrtimer_start [Tracepoint event] timer:itimer_expire [Tracepoint event] timer:itimer_state [Tracepoint event] timer:tick_stop [Tracepoint event] timer:timer_cancel [Tracepoint event] timer:timer_expire_entry [Tracepoint event] timer:timer_expire_exit [Tracepoint event] timer:timer_init [Tracepoint event] timer:timer_start [Tracepoint event] tlb:tlb_flush [Tracepoint event] udp:udp_fail_queue_rcv_skb [Tracepoint event] vmscan:mm_shrink_slab_end [Tracepoint event] vmscan:mm_shrink_slab_start [Tracepoint event] vmscan:mm_vmscan_direct_reclaim_begin [Tracepoint event] vmscan:mm_vmscan_direct_reclaim_end [Tracepoint event] vmscan:mm_vmscan_kswapd_sleep [Tracepoint event] vmscan:mm_vmscan_kswapd_wake [Tracepoint event] vmscan:mm_vmscan_lru_isolate [Tracepoint event] vmscan:mm_vmscan_lru_shrink_inactive [Tracepoint event] vmscan:mm_vmscan_memcg_isolate [Tracepoint event] vmscan:mm_vmscan_memcg_reclaim_begin [Tracepoint event] vmscan:mm_vmscan_memcg_reclaim_end [Tracepoint event] vmscan:mm_vmscan_memcg_softlimit_reclaim_begin [Tracepoint event] vmscan:mm_vmscan_memcg_softlimit_reclaim_end [Tracepoint event] vmscan:mm_vmscan_wakeup_kswapd [Tracepoint event] vmscan:mm_vmscan_writepage [Tracepoint event] vsyscall:emulate_vsyscall [Tracepoint event] workqueue:workqueue_activate_work [Tracepoint event] workqueue:workqueue_execute_end [Tracepoint event] workqueue:workqueue_execute_start [Tracepoint event] workqueue:workqueue_queue_work [Tracepoint event] writeback:balance_dirty_pages [Tracepoint event] writeback:bdi_dirty_ratelimit [Tracepoint event] writeback:global_dirty_state [Tracepoint event] writeback:wbc_writepage [Tracepoint event] writeback:writeback_bdi_register [Tracepoint event] writeback:writeback_congestion_wait [Tracepoint event] writeback:writeback_dirty_inode [Tracepoint event] writeback:writeback_dirty_inode_enqueue [Tracepoint event] writeback:writeback_dirty_inode_start [Tracepoint event] writeback:writeback_dirty_page [Tracepoint event] writeback:writeback_exec [Tracepoint event] writeback:writeback_lazytime [Tracepoint event] writeback:writeback_lazytime_iput [Tracepoint event] writeback:writeback_mark_inode_dirty [Tracepoint event] writeback:writeback_nowork [Tracepoint event] writeback:writeback_pages_written [Tracepoint event] writeback:writeback_queue [Tracepoint event] writeback:writeback_queue_io [Tracepoint event] writeback:writeback_sb_inodes_requeue [Tracepoint event] writeback:writeback_single_inode [Tracepoint event] writeback:writeback_single_inode_start [Tracepoint event] writeback:writeback_start [Tracepoint event] writeback:writeback_wait [Tracepoint event] writeback:writeback_wait_iff_congested [Tracepoint event] writeback:writeback_wake_background [Tracepoint event] writeback:writeback_write_inode [Tracepoint event] writeback:writeback_write_inode_start [Tracepoint event] writeback:writeback_written [Tracepoint event] xen:xen_cpu_load_idt [Tracepoint event] xen:xen_cpu_set_ldt [Tracepoint event] xen:xen_cpu_write_gdt_entry [Tracepoint event] xen:xen_cpu_write_idt_entry [Tracepoint event] xen:xen_cpu_write_ldt_entry [Tracepoint event] xen:xen_mc_batch [Tracepoint event] xen:xen_mc_callback [Tracepoint event] xen:xen_mc_entry [Tracepoint event] xen:xen_mc_entry_alloc [Tracepoint event] xen:xen_mc_extend_args [Tracepoint event] xen:xen_mc_flush [Tracepoint event] xen:xen_mc_flush_reason [Tracepoint event] xen:xen_mc_issue [Tracepoint event] xen:xen_mmu_alloc_ptpage [Tracepoint event] xen:xen_mmu_flush_tlb [Tracepoint event] xen:xen_mmu_flush_tlb_all [Tracepoint event] xen:xen_mmu_flush_tlb_others [Tracepoint event] xen:xen_mmu_flush_tlb_single [Tracepoint event] xen:xen_mmu_pgd_clear [Tracepoint event] xen:xen_mmu_pgd_pin [Tracepoint event] xen:xen_mmu_pgd_unpin [Tracepoint event] xen:xen_mmu_pmd_clear [Tracepoint event] xen:xen_mmu_pte_clear [Tracepoint event] xen:xen_mmu_ptep_modify_prot_commit [Tracepoint event] xen:xen_mmu_ptep_modify_prot_start [Tracepoint event] xen:xen_mmu_pud_clear [Tracepoint event] xen:xen_mmu_release_ptpage [Tracepoint event] xen:xen_mmu_set_domain_pte [Tracepoint event] xen:xen_mmu_set_pgd [Tracepoint event] xen:xen_mmu_set_pmd [Tracepoint event] xen:xen_mmu_set_pte [Tracepoint event] xen:xen_mmu_set_pte_at [Tracepoint event] xen:xen_mmu_set_pte_atomic [Tracepoint event] xen:xen_mmu_set_pud [Tracepoint event] xen:xen_mmu_write_cr3 [Tracepoint event] xfs:xfs_agf [Tracepoint event] xfs:xfs_ail_delete [Tracepoint event] xfs:xfs_ail_flushing [Tracepoint event] xfs:xfs_ail_insert [Tracepoint event] xfs:xfs_ail_locked [Tracepoint event] xfs:xfs_ail_move [Tracepoint event] xfs:xfs_ail_pinned [Tracepoint event] xfs:xfs_ail_push [Tracepoint event] xfs:xfs_alloc_exact_done [Tracepoint event] xfs:xfs_alloc_exact_error [Tracepoint event] xfs:xfs_alloc_exact_notfound [Tracepoint event] xfs:xfs_alloc_file_space [Tracepoint event] xfs:xfs_alloc_near_busy [Tracepoint event] xfs:xfs_alloc_near_error [Tracepoint event] xfs:xfs_alloc_near_first [Tracepoint event] xfs:xfs_alloc_near_greater [Tracepoint event] xfs:xfs_alloc_near_lesser [Tracepoint event] xfs:xfs_alloc_near_noentry [Tracepoint event] xfs:xfs_alloc_near_nominleft [Tracepoint event] xfs:xfs_alloc_read_agf [Tracepoint event] xfs:xfs_alloc_size_busy [Tracepoint event] xfs:xfs_alloc_size_done [Tracepoint event] xfs:xfs_alloc_size_error [Tracepoint event] xfs:xfs_alloc_size_neither [Tracepoint event] xfs:xfs_alloc_size_noentry [Tracepoint event] xfs:xfs_alloc_size_nominleft [Tracepoint event] xfs:xfs_alloc_small_done [Tracepoint event] xfs:xfs_alloc_small_error [Tracepoint event] xfs:xfs_alloc_small_freelist [Tracepoint event] xfs:xfs_alloc_small_notenough [Tracepoint event] xfs:xfs_alloc_vextent_allfailed [Tracepoint event] xfs:xfs_alloc_vextent_badargs [Tracepoint event] xfs:xfs_alloc_vextent_loopfailed [Tracepoint event] xfs:xfs_alloc_vextent_noagbp [Tracepoint event] xfs:xfs_alloc_vextent_nofix [Tracepoint event] xfs:xfs_attr_fillstate [Tracepoint event] xfs:xfs_attr_leaf_add [Tracepoint event] xfs:xfs_attr_leaf_add_new [Tracepoint event] xfs:xfs_attr_leaf_add_old [Tracepoint event] xfs:xfs_attr_leaf_add_work [Tracepoint event] xfs:xfs_attr_leaf_addname [Tracepoint event] xfs:xfs_attr_leaf_clearflag [Tracepoint event] xfs:xfs_attr_leaf_compact [Tracepoint event] xfs:xfs_attr_leaf_create [Tracepoint event] xfs:xfs_attr_leaf_flipflags [Tracepoint event] xfs:xfs_attr_leaf_get [Tracepoint event] xfs:xfs_attr_leaf_list [Tracepoint event] xfs:xfs_attr_leaf_lookup [Tracepoint event] xfs:xfs_attr_leaf_rebalance [Tracepoint event] xfs:xfs_attr_leaf_remove [Tracepoint event] xfs:xfs_attr_leaf_removename [Tracepoint event] xfs:xfs_attr_leaf_replace [Tracepoint event] xfs:xfs_attr_leaf_setflag [Tracepoint event] xfs:xfs_attr_leaf_split [Tracepoint event] xfs:xfs_attr_leaf_split_after [Tracepoint event] xfs:xfs_attr_leaf_split_before [Tracepoint event] xfs:xfs_attr_leaf_to_node [Tracepoint event] xfs:xfs_attr_leaf_to_sf [Tracepoint event] xfs:xfs_attr_leaf_toosmall [Tracepoint event] xfs:xfs_attr_leaf_unbalance [Tracepoint event] xfs:xfs_attr_list_add [Tracepoint event] xfs:xfs_attr_list_full [Tracepoint event] xfs:xfs_attr_list_leaf [Tracepoint event] xfs:xfs_attr_list_leaf_end [Tracepoint event] xfs:xfs_attr_list_node_descend [Tracepoint event] xfs:xfs_attr_list_notfound [Tracepoint event] xfs:xfs_attr_list_sf [Tracepoint event] xfs:xfs_attr_list_sf_all [Tracepoint event] xfs:xfs_attr_list_wrong_blk [Tracepoint event] xfs:xfs_attr_node_addname [Tracepoint event] xfs:xfs_attr_node_get [Tracepoint event] xfs:xfs_attr_node_list [Tracepoint event] xfs:xfs_attr_node_lookup [Tracepoint event] xfs:xfs_attr_node_removename [Tracepoint event] xfs:xfs_attr_node_replace [Tracepoint event] xfs:xfs_attr_refillstate [Tracepoint event] xfs:xfs_attr_rmtval_get [Tracepoint event] xfs:xfs_attr_rmtval_remove [Tracepoint event] xfs:xfs_attr_rmtval_set [Tracepoint event] xfs:xfs_attr_sf_add [Tracepoint event] xfs:xfs_attr_sf_addname [Tracepoint event] xfs:xfs_attr_sf_create [Tracepoint event] xfs:xfs_attr_sf_lookup [Tracepoint event] xfs:xfs_attr_sf_remove [Tracepoint event] xfs:xfs_attr_sf_removename [Tracepoint event] xfs:xfs_attr_sf_to_leaf [Tracepoint event] xfs:xfs_bdstrat_shut [Tracepoint event] xfs:xfs_bmap_post_update [Tracepoint event] xfs:xfs_bmap_pre_update [Tracepoint event] xfs:xfs_btree_corrupt [Tracepoint event] xfs:xfs_buf_bawrite [Tracepoint event] xfs:xfs_buf_delwri_queue [Tracepoint event] xfs:xfs_buf_delwri_queued [Tracepoint event] xfs:xfs_buf_delwri_split [Tracepoint event] xfs:xfs_buf_error_relse [Tracepoint event] xfs:xfs_buf_find [Tracepoint event] xfs:xfs_buf_free [Tracepoint event] xfs:xfs_buf_get [Tracepoint event] xfs:xfs_buf_get_uncached [Tracepoint event] xfs:xfs_buf_hold [Tracepoint event] xfs:xfs_buf_init [Tracepoint event] xfs:xfs_buf_iodone [Tracepoint event] xfs:xfs_buf_ioerror [Tracepoint event] xfs:xfs_buf_iowait [Tracepoint event] xfs:xfs_buf_iowait_done [Tracepoint event] xfs:xfs_buf_item_committed [Tracepoint event] xfs:xfs_buf_item_format [Tracepoint event] xfs:xfs_buf_item_format_ordered [Tracepoint event] xfs:xfs_buf_item_format_stale [Tracepoint event] xfs:xfs_buf_item_iodone [Tracepoint event] xfs:xfs_buf_item_iodone_async [Tracepoint event] xfs:xfs_buf_item_ordered [Tracepoint event] xfs:xfs_buf_item_pin [Tracepoint event] xfs:xfs_buf_item_push [Tracepoint event] xfs:xfs_buf_item_relse [Tracepoint event] xfs:xfs_buf_item_size [Tracepoint event] xfs:xfs_buf_item_size_ordered [Tracepoint event] xfs:xfs_buf_item_size_stale [Tracepoint event] xfs:xfs_buf_item_unlock [Tracepoint event] xfs:xfs_buf_item_unlock_stale [Tracepoint event] xfs:xfs_buf_item_unpin [Tracepoint event] xfs:xfs_buf_item_unpin_stale [Tracepoint event] xfs:xfs_buf_lock [Tracepoint event] xfs:xfs_buf_lock_done [Tracepoint event] xfs:xfs_buf_read [Tracepoint event] xfs:xfs_buf_rele [Tracepoint event] xfs:xfs_buf_submit [Tracepoint event] xfs:xfs_buf_submit_wait [Tracepoint event] xfs:xfs_buf_trylock [Tracepoint event] xfs:xfs_buf_unlock [Tracepoint event] xfs:xfs_buf_wait_buftarg [Tracepoint event] xfs:xfs_bunmap [Tracepoint event] xfs:xfs_collapse_file_space [Tracepoint event] xfs:xfs_create [Tracepoint event] xfs:xfs_da_btree_corrupt [Tracepoint event] xfs:xfs_da_fixhashpath [Tracepoint event] xfs:xfs_da_grow_inode [Tracepoint event] xfs:xfs_da_join [Tracepoint event] xfs:xfs_da_link_after [Tracepoint event] xfs:xfs_da_link_before [Tracepoint event] xfs:xfs_da_node_add [Tracepoint event] xfs:xfs_da_node_create [Tracepoint event] xfs:xfs_da_node_rebalance [Tracepoint event] xfs:xfs_da_node_remove [Tracepoint event] xfs:xfs_da_node_split [Tracepoint event] xfs:xfs_da_node_toosmall [Tracepoint event] xfs:xfs_da_node_unbalance [Tracepoint event] xfs:xfs_da_path_shift [Tracepoint event] xfs:xfs_da_root_join [Tracepoint event] xfs:xfs_da_root_split [Tracepoint event] xfs:xfs_da_shrink_inode [Tracepoint event] xfs:xfs_da_split [Tracepoint event] xfs:xfs_da_swap_lastblock [Tracepoint event] xfs:xfs_da_unlink_back [Tracepoint event] xfs:xfs_da_unlink_forward [Tracepoint event] xfs:xfs_delalloc_enospc [Tracepoint event] xfs:xfs_destroy_inode [Tracepoint event] xfs:xfs_dir2_block_addname [Tracepoint event] xfs:xfs_dir2_block_lookup [Tracepoint event] xfs:xfs_dir2_block_removename [Tracepoint event] xfs:xfs_dir2_block_replace [Tracepoint event] xfs:xfs_dir2_block_to_leaf [Tracepoint event] xfs:xfs_dir2_block_to_sf [Tracepoint event] xfs:xfs_dir2_grow_inode [Tracepoint event] xfs:xfs_dir2_leaf_addname [Tracepoint event] xfs:xfs_dir2_leaf_lookup [Tracepoint event] xfs:xfs_dir2_leaf_removename [Tracepoint event] xfs:xfs_dir2_leaf_replace [Tracepoint event] xfs:xfs_dir2_leaf_to_block [Tracepoint event] xfs:xfs_dir2_leaf_to_node [Tracepoint event] xfs:xfs_dir2_leafn_add [Tracepoint event] xfs:xfs_dir2_leafn_moveents [Tracepoint event] xfs:xfs_dir2_leafn_remove [Tracepoint event] xfs:xfs_dir2_node_addname [Tracepoint event] xfs:xfs_dir2_node_lookup [Tracepoint event] xfs:xfs_dir2_node_removename [Tracepoint event] xfs:xfs_dir2_node_replace [Tracepoint event] xfs:xfs_dir2_node_to_leaf [Tracepoint event] xfs:xfs_dir2_sf_addname [Tracepoint event] xfs:xfs_dir2_sf_create [Tracepoint event] xfs:xfs_dir2_sf_lookup [Tracepoint event] xfs:xfs_dir2_sf_removename [Tracepoint event] xfs:xfs_dir2_sf_replace [Tracepoint event] xfs:xfs_dir2_sf_to_block [Tracepoint event] xfs:xfs_dir2_sf_toino4 [Tracepoint event] xfs:xfs_dir2_sf_toino8 [Tracepoint event] xfs:xfs_dir2_shrink_inode [Tracepoint event] xfs:xfs_dir_fsync [Tracepoint event] xfs:xfs_discard_busy [Tracepoint event] xfs:xfs_discard_exclude [Tracepoint event] xfs:xfs_discard_extent [Tracepoint event] xfs:xfs_discard_toosmall [Tracepoint event] xfs:xfs_dqadjust [Tracepoint event] xfs:xfs_dqalloc [Tracepoint event] xfs:xfs_dqattach_found [Tracepoint event] xfs:xfs_dqattach_get [Tracepoint event] xfs:xfs_dqflush [Tracepoint event] xfs:xfs_dqflush_done [Tracepoint event] xfs:xfs_dqflush_force [Tracepoint event] xfs:xfs_dqget_dup [Tracepoint event] xfs:xfs_dqget_freeing [Tracepoint event] xfs:xfs_dqget_hit [Tracepoint event] xfs:xfs_dqget_miss [Tracepoint event] xfs:xfs_dqput [Tracepoint event] xfs:xfs_dqput_free [Tracepoint event] xfs:xfs_dqput_wait [Tracepoint event] xfs:xfs_dqread [Tracepoint event] xfs:xfs_dqread_fail [Tracepoint event] xfs:xfs_dqreclaim_busy [Tracepoint event] xfs:xfs_dqreclaim_dirty [Tracepoint event] xfs:xfs_dqreclaim_done [Tracepoint event] xfs:xfs_dqreclaim_want [Tracepoint event] xfs:xfs_dqrele [Tracepoint event] xfs:xfs_dqtobp_read [Tracepoint event] xfs:xfs_dquot_dqalloc [Tracepoint event] xfs:xfs_dquot_dqdetach [Tracepoint event] xfs:xfs_evict_inode [Tracepoint event] xfs:xfs_extent_busy [Tracepoint event] xfs:xfs_extent_busy_clear [Tracepoint event] xfs:xfs_extent_busy_enomem [Tracepoint event] xfs:xfs_extent_busy_force [Tracepoint event] xfs:xfs_extent_busy_reuse [Tracepoint event] xfs:xfs_extent_busy_trim [Tracepoint event] xfs:xfs_extlist [Tracepoint event] xfs:xfs_file_buffered_write [Tracepoint event] xfs:xfs_file_compat_ioctl [Tracepoint event] xfs:xfs_file_direct_write [Tracepoint event] xfs:xfs_file_fsync [Tracepoint event] xfs:xfs_file_ioctl [Tracepoint event] xfs:xfs_file_read [Tracepoint event] xfs:xfs_file_splice_read [Tracepoint event] xfs:xfs_filemap_fault [Tracepoint event] xfs:xfs_filemap_page_mkwrite [Tracepoint event] xfs:xfs_filestream_free [Tracepoint event] xfs:xfs_filestream_lookup [Tracepoint event] xfs:xfs_filestream_pick [Tracepoint event] xfs:xfs_filestream_scan [Tracepoint event] xfs:xfs_free_extent [Tracepoint event] xfs:xfs_free_file_space [Tracepoint event] xfs:xfs_gbmap_direct [Tracepoint event] xfs:xfs_gbmap_direct_endio [Tracepoint event] xfs:xfs_gbmap_direct_new [Tracepoint event] xfs:xfs_gbmap_direct_none [Tracepoint event] xfs:xfs_gbmap_direct_update [Tracepoint event] xfs:xfs_get_acl [Tracepoint event] xfs:xfs_get_blocks_alloc [Tracepoint event] xfs:xfs_get_blocks_found [Tracepoint event] xfs:xfs_get_blocks_notfound [Tracepoint event] xfs:xfs_getattr [Tracepoint event] xfs:xfs_ialloc_read_agi [Tracepoint event] xfs:xfs_iext_insert [Tracepoint event] xfs:xfs_iext_remove [Tracepoint event] xfs:xfs_iget_hit [Tracepoint event] xfs:xfs_iget_miss [Tracepoint event] xfs:xfs_iget_reclaim [Tracepoint event] xfs:xfs_iget_reclaim_fail [Tracepoint event] xfs:xfs_iget_skip [Tracepoint event] xfs:xfs_ihold [Tracepoint event] xfs:xfs_ilock [Tracepoint event] xfs:xfs_ilock_demote [Tracepoint event] xfs:xfs_ilock_nowait [Tracepoint event] xfs:xfs_inactive_symlink [Tracepoint event] xfs:xfs_inode_clear_eofblocks_tag [Tracepoint event] xfs:xfs_inode_free_eofblocks_invalid [Tracepoint event] xfs:xfs_inode_item_push [Tracepoint event] xfs:xfs_inode_pin [Tracepoint event] xfs:xfs_inode_set_eofblocks_tag [Tracepoint event] xfs:xfs_inode_unpin [Tracepoint event] xfs:xfs_inode_unpin_nowait [Tracepoint event] xfs:xfs_insert_file_space [Tracepoint event] xfs:xfs_invalidatepage [Tracepoint event] xfs:xfs_ioctl_setattr [Tracepoint event] xfs:xfs_iomap_prealloc_size [Tracepoint event] xfs:xfs_irec_merge_post [Tracepoint event] xfs:xfs_irec_merge_pre [Tracepoint event] xfs:xfs_irele [Tracepoint event] xfs:xfs_itruncate_extents_end [Tracepoint event] xfs:xfs_itruncate_extents_start [Tracepoint event] xfs:xfs_iunlock [Tracepoint event] xfs:xfs_link [Tracepoint event] xfs:xfs_log_assign_tail_lsn [Tracepoint event] xfs:xfs_log_done_nonperm [Tracepoint event] xfs:xfs_log_done_perm [Tracepoint event] xfs:xfs_log_force [Tracepoint event] xfs:xfs_log_grant_sleep [Tracepoint event] xfs:xfs_log_grant_wake [Tracepoint event] xfs:xfs_log_grant_wake_up [Tracepoint event] xfs:xfs_log_recover_buf_cancel [Tracepoint event] xfs:xfs_log_recover_buf_cancel_add [Tracepoint event] xfs:xfs_log_recover_buf_cancel_ref_inc [Tracepoint event] xfs:xfs_log_recover_buf_dquot_buf [Tracepoint event] xfs:xfs_log_recover_buf_inode_buf [Tracepoint event] xfs:xfs_log_recover_buf_not_cancel [Tracepoint event] xfs:xfs_log_recover_buf_recover [Tracepoint event] xfs:xfs_log_recover_buf_reg_buf [Tracepoint event] xfs:xfs_log_recover_inode_cancel [Tracepoint event] xfs:xfs_log_recover_inode_recover [Tracepoint event] xfs:xfs_log_recover_inode_skip [Tracepoint event] xfs:xfs_log_recover_item_add [Tracepoint event] xfs:xfs_log_recover_item_add_cont [Tracepoint event] xfs:xfs_log_recover_item_recover [Tracepoint event] xfs:xfs_log_recover_item_reorder_head [Tracepoint event] xfs:xfs_log_recover_item_reorder_tail [Tracepoint event] xfs:xfs_log_regrant [Tracepoint event] xfs:xfs_log_regrant_exit [Tracepoint event] xfs:xfs_log_regrant_reserve_enter [Tracepoint event] xfs:xfs_log_regrant_reserve_exit [Tracepoint event] xfs:xfs_log_regrant_reserve_sub [Tracepoint event] xfs:xfs_log_reserve [Tracepoint event] xfs:xfs_log_reserve_exit [Tracepoint event] xfs:xfs_log_umount_write [Tracepoint event] xfs:xfs_log_ungrant_enter [Tracepoint event] xfs:xfs_log_ungrant_exit [Tracepoint event] xfs:xfs_log_ungrant_sub [Tracepoint event] xfs:xfs_lookup [Tracepoint event] xfs:xfs_map_blocks_alloc [Tracepoint event] xfs:xfs_map_blocks_found [Tracepoint event] xfs:xfs_pagecache_inval [Tracepoint event] xfs:xfs_perag_clear_eofblocks [Tracepoint event] xfs:xfs_perag_clear_reclaim [Tracepoint event] xfs:xfs_perag_get [Tracepoint event] xfs:xfs_perag_get_tag [Tracepoint event] xfs:xfs_perag_put [Tracepoint event] xfs:xfs_perag_set_eofblocks [Tracepoint event] xfs:xfs_perag_set_reclaim [Tracepoint event] xfs:xfs_read_agf [Tracepoint event] xfs:xfs_read_agi [Tracepoint event] xfs:xfs_readdir [Tracepoint event] xfs:xfs_readlink [Tracepoint event] xfs:xfs_releasepage [Tracepoint event] xfs:xfs_remove [Tracepoint event] xfs:xfs_rename [Tracepoint event] xfs:xfs_reset_dqcounts [Tracepoint event] xfs:xfs_setattr [Tracepoint event] xfs:xfs_setfilesize [Tracepoint event] xfs:xfs_swap_extent_after [Tracepoint event] xfs:xfs_swap_extent_before [Tracepoint event] xfs:xfs_symlink [Tracepoint event] xfs:xfs_trans_bhold [Tracepoint event] xfs:xfs_trans_bhold_release [Tracepoint event] xfs:xfs_trans_binval [Tracepoint event] xfs:xfs_trans_bjoin [Tracepoint event] xfs:xfs_trans_brelse [Tracepoint event] xfs:xfs_trans_buf_ordered [Tracepoint event] xfs:xfs_trans_commit_lsn [Tracepoint event] xfs:xfs_trans_get_buf [Tracepoint event] xfs:xfs_trans_get_buf_recur [Tracepoint event] xfs:xfs_trans_getsb [Tracepoint event] xfs:xfs_trans_getsb_recur [Tracepoint event] xfs:xfs_trans_log_buf [Tracepoint event] xfs:xfs_trans_read_buf [Tracepoint event] xfs:xfs_trans_read_buf_io [Tracepoint event] xfs:xfs_trans_read_buf_recur [Tracepoint event] xfs:xfs_trans_read_buf_shut [Tracepoint event] xfs:xfs_unwritten_convert [Tracepoint event] xfs:xfs_update_time [Tracepoint event] xfs:xfs_vm_bmap [Tracepoint event] xfs:xfs_writepage [Tracepoint event] xfs:xfs_zero_file_space [Tracepoint event] xhci-hcd:xhci_address_ctx [Tracepoint event] xhci-hcd:xhci_cmd_completion [Tracepoint event] xhci-hcd:xhci_dbg_address [Tracepoint event] xhci-hcd:xhci_dbg_cancel_urb [Tracepoint event] xhci-hcd:xhci_dbg_context_change [Tracepoint event] xhci-hcd:xhci_dbg_init [Tracepoint event] xhci-hcd:xhci_dbg_quirks [Tracepoint event] xhci-hcd:xhci_dbg_reset_ep [Tracepoint event] xhci-hcd:xhci_dbg_ring_expansion [Tracepoint event]

    opened by edwintorok 123
  • Hangs in MOZ_CRASH

    Hangs in MOZ_CRASH

    kernel version: 4.7.2 (x86-64) rr version: d3a9a2a8e8e7aa29426936f951b51a5c49dff049 gcc version: 6.2.0 gdb version: 7.11.1 m-c version: 8c9c4e816e86f903c1d820f3f29715dc070a5a4a

    When it hits MOZ_CRASH in content process, rr, firefox and plugin-container are all hangs. Backtrace in rr:

    #0  0x00007f79146b5b7b in waitpid () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x00000000004fdd56 in rr::Task::wait (this=this@entry=0x11a4610, interrupt_after_elapsed=interrupt_after_elapsed@entry=0) at /home/wcpan/local/src/rr/src/Task.cc:1144
#2  0x00000000004ffb77 in rr::Task::resume_execution (this=this@entry=0x11a4610, how=how@entry=rr::RESUME_SINGLESTEP, wait_how=wait_how@entry=rr::RESUME_WAIT, 
    tick_period=tick_period@entry=rr::RESUME_NO_TICKS, sig=sig@entry=11) at /home/wcpan/local/src/rr/src/Task.cc:915
#3  0x0000000000493353 in rr::inject_handled_signal (t=0x11a4610) at /home/wcpan/local/src/rr/src/RecordSession.cc:1038
#4  rr::RecordSession::signal_state_changed (this=this@entry=0x1128b50, t=t@entry=0x11a4610, step_state=step_state@entry=0x7ffc52d289d0)
    at /home/wcpan/local/src/rr/src/RecordSession.cc:1127
#5  0x000000000049679a in rr::RecordSession::record_step (this=0x1128b50) at /home/wcpan/local/src/rr/src/RecordSession.cc:1779
#6  0x000000000048d45e in rr::record (flags=..., args=...) at /home/wcpan/local/src/rr/src/RecordCommand.cc:296
#7  rr::RecordCommand::run (this=0x75d800 <rr::RecordCommand::singleton>, args=...) at /home/wcpan/local/src/rr/src/RecordCommand.cc:363
#8  0x0000000000430dff in main (argc=<optimized out>, argv=<optimized out>) at /home/wcpan/local/src/rr/src/main.cc:270
    opened by legnaleurc 81
  • Assertion !syscall_bp_vm failed to hold during replay

    Assertion !syscall_bp_vm failed to hold during replay

    In https://github.com/gsliepen/tinc/issues/252 I attempt to use rr record and rr replay to debug a vpn server process that's crashing somewhere. The stack gets smashed so I hoped I could use rr to rewind and make sense of things.

    When trying to replay the recording I run in to the error in appendix 1 of the above, repasted here:

    $ rr replay /var/lib/containers/gate/root/.local/share/rr/tincd-7                                                                                         
GNU gdb (GDB) 9.2                                                                                                                                                                                                                                                              
Copyright (C) 2020 Free Software Foundation, Inc.                                                                                                                                                                                                                              License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>                                                          
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.                                                                                  
Type "show copying" and "show warranty" for details.                                                                                                                                                                                                                           
This GDB was configured as "x86_64-unknown-linux-gnu".                                                                                                                                                                                                                         Type "show configuration" for configuration details.                                                                                   
For bug reporting instructions, please see:                                                                                            
<http://www.gnu.org/software/gdb/bugs/>.                                                                                               
Find the GDB manual and other documentation resources online at:                                                                                                                                                                                                               
    <http://www.gnu.org/software/gdb/documentation/>.                                                                                  
                                                                                                                                       
For help, type "help".                              
Type "apropos word" to search for commands related to "word"...                                                                        
Reading symbols from /nix/store/3218jrqqrh8cbbnhijhh2vxw2m9vm85m-tinc-1.1pre17/bin/tincd...                                                                                                                                                                                    
Really redefine built-in command "restart"? (y or n) [answered Y; input not from terminal]                                             
Remote debugging using 127.0.0.1:27767                                                                                                 
Reading symbols from /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/ld-linux-x86-64.so.2...                                
(No debugging symbols found in /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/ld-linux-x86-64.so.2)                        
0x00007f02c7d3d090 in _start () from /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/ld-linux-x86-64.so.2                                                                                                                                                           
(rr) c                                                                                                                                 
Continuing.                                                                                                                            
[FATAL /build/source/src/ReplaySession.cc:570:enter_syscall()]                                                                         
 (task 27768 (rec:406) at time 385)                                                                                                    
 -> Assertion `!syscall_bp_vm' failed to hold.                                                                                                                                                                                                                                 
Tail of trace dump:                                                                                                                    
{                                                                                                                                                                                                                                                                              
  real_time:1450867.692367 global_time:365, event:`INSTRUCTION_TRAP' tid:406, ticks:140528                                        
rax:0x1c004121 rbx:0x1c0003f rcx:0x3f rdx:0x0 rsi:0xf0b5ff rdi:0xbc rbp:0x3 rsp:0x7ffeeb8176c0 r8:0x7f02c7d65698 r9:0x7f02c7d6569c r10:0x0 r11:0x7f02c784d710 r12:0x7ffeeb81770f r13:0x7f02c784d5e0 r14:0x7f02c7d655c0 r15:0x7ffeeb81770f rip:0x7f02c7775183 eflags:0x10246 cs:
0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.692623 global_time:366, event:`INSTRUCTION_TRAP' tid:406, ticks:140535
rax:0x76036301 rbx:0xf0b5ff rcx:0x0 rdx:0xc30000 rsi:0x0 rdi:0xbf rbp:0xbf rsp:0x7ffeeb8176f0 r8:0x4 r9:0x7f02c7d65601 r10:0x0 r11:0x7f02c784d710 r12:0x7ffeeb81770e r13:0x1 r14:0x7f02c7d655c0 r15:0x7ffeeb81770f rip:0x7f02c777534a eflags:0x10202 cs:0x33 ss:0x2b ds:0x0 es:
0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.692787 global_time:367, event:`INSTRUCTION_TRAP' tid:406, ticks:140640
rax:0x1c004121 rbx:0x1c0003f rcx:0x3f rdx:0x0 rsi:0xf0b5ff rdi:0xbf rbp:0x6 rsp:0x7ffeeb8176c0 r8:0x7f02c7d65698 r9:0x7f02c7d6569c r10:0x0 r11:0x7f02c784d710 r12:0x7ffeeb81770f r13:0x7f02c784d5e0 r14:0x7f02c7d655c0 r15:0x7ffeeb81770f rip:0x7f02c7775183 eflags:0x10246 cs:
0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.692882 global_time:368, event:`INSTRUCTION_TRAP' tid:406, ticks:140644
rax:0x1c004122 rbx:0x1c0003f rcx:0x3f rdx:0x0 rsi:0x1 rdi:0xbf rbp:0x6 rsp:0x7ffeeb8176c0 r8:0x4 r9:0x7f02c7d65601 r10:0x0 r11:0x7f02c784d710 r12:0x7ffeeb81770f r13:0x7f02c784d5e0 r14:0x7f02c7d655c0 r15:0x7ffeeb81770f rip:0x7f02c77751ac eflags:0x10202 cs:0x33 ss:0x2b ds:
0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.692973 global_time:369, event:`INSTRUCTION_TRAP' tid:406, ticks:140649
rax:0x1c004143 rbx:0x1c0003f rcx:0x1ff rdx:0x0 rsi:0x2 rdi:0xbf rbp:0x6 rsp:0x7ffeeb8176c0 r8:0x4 r9:0x7f02c7d65601 r10:0x0 r11:0x7f02c784d710 r12:0x7ffeeb81770f r13:0x7f02c784d5e0 r14:0x7f02c7d655c0 r15:0x7ffeeb81770f rip:0x7f02c77751ac eflags:0x10202 cs:0x33 ss:0x2b ds
:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.693066 global_time:370, event:`INSTRUCTION_TRAP' tid:406, ticks:140657
rax:0x76036301 rbx:0xf0b5ff rcx:0x0 rdx:0xc30000 rsi:0x2 rdi:0xc2 rbp:0xc2 rsp:0x7ffeeb8176f0 r8:0x4 r9:0x7f02c7d65600 r10:0x0 r11:0x7f02c784d710 r12:0x7ffeeb81770e r13:0x1 r14:0x7f02c7d655c0 r15:0x7ffeeb81770f rip:0x7f02c777534a eflags:0x10202 cs:0x33 ss:0x2b ds:0x0 es:
0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.693174 global_time:371, event:`INSTRUCTION_TRAP' tid:406, ticks:140762
rax:0x1c004121 rbx:0x1c0003f rcx:0x3f rdx:0x0 rsi:0xf0b5ff rdi:0xc2 rbp:0x9 rsp:0x7ffeeb8176c0 r8:0x7f02c7d65698 r9:0x7f02c7d6569c r10:0x0 r11:0x7f02c784d710 r12:0x7ffeeb81770f r13:0x7f02c784d5e0 r14:0x7f02c7d655c0 r15:0x7ffeeb81770f rip:0x7f02c7775183 eflags:0x10246 cs:
0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.693291 global_time:372, event:`INSTRUCTION_TRAP' tid:406, ticks:140766
rax:0x1c004122 rbx:0x1c0003f rcx:0x3f rdx:0x0 rsi:0x1 rdi:0xc2 rbp:0x9 rsp:0x7ffeeb8176c0 r8:0x4 r9:0x7f02c7d65601 r10:0x0 r11:0x7f02c784d710 r12:0x7ffeeb81770f r13:0x7f02c784d5e0 r14:0x7f02c7d655c0 r15:0x7ffeeb81770f rip:0x7f02c77751ac eflags:0x10202 cs:0x33 ss:0x2b ds:
0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.693387 global_time:373, event:`INSTRUCTION_TRAP' tid:406, ticks:140771
rax:0x1c004143 rbx:0x1c0003f rcx:0x1ff rdx:0x0 rsi:0x2 rdi:0xc2 rbp:0x9 rsp:0x7ffeeb8176c0 r8:0x4 r9:0x7f02c7d65601 r10:0x0 r11:0x7f02c784d710 r12:0x7ffeeb81770f r13:0x7f02c784d5e0 r14:0x7f02c7d655c0 r15:0x7ffeeb81770f rip:0x7f02c77751ac eflags:0x10202 cs:0x33 ss:0x2b ds
:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.693490 global_time:374, event:`INSTRUCTION_TRAP' tid:406, ticks:140778
rax:0x1c03c163 rbx:0x2c0003f rcx:0xfff rdx:0x6 rsi:0x3 rdi:0xc2 rbp:0x9 rsp:0x7ffeeb8176c0 r8:0x4 r9:0x7f02c7d65600 r10:0x0 r11:0x7f02c784d710 r12:0x7ffeeb81770f r13:0x7f02c784d5e0 r14:0x7f02c7d655c0 r15:0x7ffeeb81770f rip:0x7f02c77751ac eflags:0x10206 cs:0x33 ss:0x2b ds
:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.693590 global_time:375, event:`INSTRUCTION_TRAP' tid:406, ticks:140790
rax:0x1c004121 rbx:0x1c0003f rcx:0x3f rdx:0x0 rsi:0x1 rdi:0x3 rbp:0x300000 rsp:0x7ffeeb817750 r8:0x1 r9:0x4 r10:0x0 r11:0x0 r12:0x7f02c7d655c0 r13:0x8000 r14:0x0 r15:0x0 rip:0x7f02c76f5805 eflags:0x10246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffff
ffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.693694 global_time:376, event:`INSTRUCTION_TRAP' tid:406, ticks:140794
rax:0x1c004122 rbx:0x1c0003f rcx:0x3f rdx:0x0 rsi:0x1 rdi:0x3 rbp:0x300000 rsp:0x7ffeeb817750 r8:0x2 r9:0x4 r10:0x0 r11:0x0 r12:0x7f02c7d655c0 r13:0x8000 r14:0x0 r15:0x0 rip:0x7f02c76f5805 eflags:0x10206 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffff
ffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.693802 global_time:377, event:`INSTRUCTION_TRAP' tid:406, ticks:140798
rax:0x1c004143 rbx:0x1c0003f rcx:0x1ff rdx:0x0 rsi:0x1 rdi:0x3 rbp:0x300000 rsp:0x7ffeeb817750 r8:0x3 r9:0x4 r10:0x0 r11:0x0 r12:0x7f02c7d655c0 r13:0x8000 r14:0x0 r15:0x0 rip:0x7f02c76f5805 eflags:0x10206 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xfffffffffff
fffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.693912 global_time:378, event:`INSTRUCTION_TRAP' tid:406, ticks:140802
rax:0x1c03c163 rbx:0x2c0003f rcx:0xfff rdx:0x6 rsi:0x1 rdi:0x2 rbp:0x300000 rsp:0x7ffeeb817750 r8:0x4 r9:0x4 r10:0x0 r11:0x1 r12:0x7f02c7d655c0 r13:0x8000 r14:0x0 r15:0x0 rip:0x7f02c76f5805 eflags:0x10202 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xfffffffffff
fffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.694018 global_time:379, event:`INSTRUCTION_TRAP' tid:406, ticks:140810
rax:0x1 rbx:0x2 rcx:0x100 rdx:0x1 rsi:0x1 rdi:0x0 rbp:0x300000 rsp:0x7ffeeb817750 r8:0x3 r9:0x1 r10:0xf r11:0x1 r12:0x7f02c7d655c0 r13:0x8000 r14:0x0 r15:0x0 rip:0x7f02c76f59b8 eflags:0x10206 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base
:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.694119 global_time:380, event:`INSTRUCTION_TRAP' tid:406, ticks:140815
rax:0x4 rbx:0x4 rcx:0x201 rdx:0x1 rsi:0x1 rdi:0x0 rbp:0x300000 rsp:0x7ffeeb817750 r8:0x2 r9:0x2 r10:0xf r11:0x1 r12:0x7f02c7d655c0 r13:0x8000 r14:0x0 r15:0x0 rip:0x7f02c76f59b8 eflags:0x10202 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base
:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.694289 global_time:381, event:`INSTRUCTION_TRAP' tid:406, ticks:140973
rax:0x14 rbx:0x756e6547 rcx:0x6c65746e rdx:0x49656e69 rsi:0x7ffeeb817818 rdi:0x7f02c7d07fb0 rbp:0x0 rsp:0x7ffeeb817738 r8:0x7f02c7d3a6a0 r9:0x2f r10:0x3 r11:0x2 r12:0x7ffeeb817818 r13:0x7f02c7d07fb0 r14:0x7f02c7d67130 r15:0x0 rip:0x7f02c7c48bfb eflags:0x10246 cs:0x33 ss:
0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.694404 global_time:382, event:`INSTRUCTION_TRAP' tid:406, ticks:140975
rax:0x1c004121 rbx:0x1c0003f rcx:0x3f rdx:0x0 rsi:0x7ffeeb817818 rdi:0x7f02c7d07fb0 rbp:0x0 rsp:0x7ffeeb817738 r8:0x7f02c7d3a6a0 r9:0x0 r10:0xffffffff r11:0x14 r12:0x7ffeeb817818 r13:0x7f02c7d07fb0 r14:0x7f02c7d67130 r15:0x0 rip:0x7f02c7c48cc9 eflags:0x10202 cs:0x33 ss:0
x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.694508 global_time:383, event:`INSTRUCTION_TRAP' tid:406, ticks:140975
rax:0x306d4 rbx:0x1100800 rcx:0x3ffafbbf rdx:0xbfebfbff rsi:0x7ffeeb817818 rdi:0x7f02c7d07fb0 rbp:0x0 rsp:0x7ffeeb817738 r8:0x7f02c7d3a6a0 r9:0x0 r10:0x1 r11:0x14 r12:0x7ffeeb817818 r13:0x7f02c7d07fb0 r14:0x7f02c7d67130 r15:0x0 rip:0x7f02c7c48cde eflags:0x10202 cs:0x33 s
s:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.694609 global_time:384, event:`INSTRUCTION_TRAP' tid:406, ticks:140984
rax:0x0 rbx:0x21827ab rcx:0x0 rdx:0x0 rsi:0x7ffeeb817818 rdi:0x7f02c7d07fb0 rbp:0x0 rsp:0x7ffeeb817738 r8:0x7f02c7d3a6a0 r9:0x3ffaf3bf r10:0xffebfbff r11:0x14 r12:0x7ffeeb817818 r13:0x7f02c7d07fb0 r14:0x7f02c7d67130 r15:0x0 rip:0x7f02c7c48d68 eflags:0x10246 cs:0x33 ss:0x
2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.694787 global_time:385, event:`SYSCALL: futex' (state:ENTERING_SYSCALL) tid:406, ticks:141106
rax:0xffffffffffffffda rbx:0x7f02c79d8048 rcx:0xffffffffffffffff rdx:0x7fffffff rsi:0x81 rdi:0x7f02c79d8048 rbp:0x7f02c79d54e0 rsp:0x7ffeeb817540 r8:0x7f02c76cd080 r9:0x1 r10:0x0 r11:0x246 r12:0x7ffeeb817540 r13:0x7ffeeb817868 r14:0x7f02c7d32da8 r15:0x0 rip:0x7f02c76c141
e eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xca fs_base:0x7f02c76afe00 gs_base:0x0
}
{
  real_time:1450867.694882 global_time:386, event:`SYSCALL: futex' (state:EXITING_SYSCALL) tid:406, ticks:141106
rax:0x0 rbx:0x7f02c79d8048 rcx:0xffffffffffffffff rdx:0x7fffffff rsi:0x81 rdi:0x7f02c79d8048 rbp:0x7f02c79d54e0 rsp:0x7ffeeb817540 r8:0x7f02c76cd080 r9:0x1 r10:0x0 r11:0x246 r12:0x7ffeeb817540 r13:0x7ffeeb817868 r14:0x7f02c7d32da8 r15:0x0 rip:0x7f02c76c141e eflags:0x246 
cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xca fs_base:0x7f02c76afe00 gs_base:0x0
}
=== Start rr backtrace:
rr(_ZN2rr13dump_rr_stackEv+0x44)[0x89f462]
rr(_ZN2rr9GdbServer15emergency_debugEPNS_4TaskE+0x1a2)[0x71d802]
rr[0x743fa0]
rr(_ZN2rr21EmergencyDebugOstreamD1Ev+0x62)[0x7441fa]
rr(_ZN2rr13ReplaySession13enter_syscallEPNS_10ReplayTaskERKNS0_15StepConstraintsE+0x6d3)[0x8033d7]
rr(_ZN2rr13ReplaySession18try_one_trace_stepEPNS_10ReplayTaskERKNS0_15StepConstraintsE+0xf5)[0x806f23]
rr(_ZN2rr13ReplaySession11replay_stepERKNS0_15StepConstraintsE+0x15b)[0x80830d]
rr(_ZN2rr14ReplayTimeline19replay_step_forwardENS_10RunCommandEl+0xe9)[0x823e61]
rr(_ZN2rr9GdbServer14debug_one_stepERNS_10GdbRequestE+0x555)[0x71acbf]
rr(_ZN2rr9GdbServer12serve_replayERKNS0_15ConnectionFlagsE+0x4b9)[0x71c993]
rr[0x7fe95f]
rr(_ZN2rr13ReplayCommand3runERSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x416)[0x7ff3a0]
rr(main+0x227)[0x8b7a7b]
/nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6(__libc_start_main+0xed)[0x7f15e72a1c7d]
rr(_start+0x2a)[0x68c4da]
=== End rr backtrace
Launch gdb with
  gdb '-l' '10000' '-ex' 'set sysroot /' '-ex' 'target extended-remote 127.0.0.1:27768' /nix/store/3218jrqqrh8cbbnhijhh2vxw2m9vm85m-tinc-1.1pre17/bin/tincd

    I'm not comfortable sharing the trace because it should contain vpn key material and whatnot. I'm not able to arrange a new recording because it'll be a few days till I can get back in the situation that causes this failure.

    rr replay -a also fails.

    Is it possible to disable the assertion or something and look at the replay anyway? What does the assert failure mean? I really just want a backtrace of where things start going wrong in tinc.

    opened by deliciouslytyped 78
  • Unknown ioctl(0xc0406400) on Chromium

    Unknown ioctl(0xc0406400) on Chromium

    I tried using rr on Chromium, and I got this:

    [FATAL /home/andreu/Work/tools/rr/src/record_syscall.cc:6123:rec_process_syscall_arch()] 
 (task 32093 (rec:32093) at time 53465)
 -> Assertion `t->regs().syscall_result_signed() == -syscall_state.expect_errno' failed to hold. Expected EINVAL for 'ioctl' but got result 0 (errno SUCCESS); Unknown ioctl(0xc0406400): type:0x64 nr:0 dir:0x3 size:64 addr:0x7fc460e4aef0

    This happens both with Fedora's packaged version of rr and with a version built from the master branch. This particular output is from Chromium's content-shell run with the --single-process --no-sandbox flags, with rr built from the main branch, but it also happens with regular Chromium, without those flags, and with Fedora's packaged version of rr 5.5.0.

    Might be related to #1412.

    opened by andreubotella 75
  • Add support for Haswell microarch

    Add support for Haswell microarch

    Users in the field have haswell machines now. I think it's a zig microarch, so we might have to do a bit of research to see what its deterministic counters are. Crossing my fingers that rbc hasn't changed in haswell.

    @rocallahan do you have access to a haswell machine, in case this gets complicated?

    opened by joneschrisg 57
  • FATAL - Replay got unexpected signal (or none) 11

    FATAL - Replay got unexpected signal (or none) 11 

    [FATAL /builddir/build/BUILD/rr-5.4.0/src/ReplaySession.cc:1306:flush_syscallbuf()]
 (task 9973 (rec:9596) at time 1037)
 -> Assertion `t->stop_sig() == SIGSEGV && r.ip() == t->vm()->do_breakpoint_fault_addr()' failed to hold. Replay got unexpected signal (or none) 11
Tail of trace dump:
{
  real_time:3649183.888775 global_time:1017, event:`SYSCALL: brk' (state:EXITING_SYSCALL) tid:9596, ticks:3759581
rax:0x257f000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x257f000 rbp:0xffffffffffffc000 rsp:0x681ffde0 r8:0x255efd0 r9:0x3 r10:0x2561000 r11:0x246 r12:0x7f9c7380b620 r13:0x7f9c73809438 r14:0x0 r15:0x7f9c73808ba0 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xc fs_base:0x7f9c74b2ab80 gs_base:0x0
  { map_file:"<ZERO>", addr:0x257f000, length:0x4000, prot_flags:"---p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x4000 }
}
{
  real_time:3649183.888822 global_time:1018, event:`SYSCALL: brk' (state:ENTERING_SYSCALL) tid:9596, ticks:3759596
rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x0 rbp:0x0 rsp:0x681ffde0 r8:0x255efd0 r9:0x3 r10:0x2561000 r11:0x246 r12:0x7f9c7380b620 r13:0x7f9c73809438 r14:0x0 r15:0x7f9c73808ba0 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xc fs_base:0x7f9c74b2ab80 gs_base:0x0
}
{
  real_time:3649183.888850 global_time:1019, event:`SYSCALL: brk' (state:EXITING_SYSCALL) tid:9596, ticks:3759596
rax:0x257f000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x0 rbp:0x0 rsp:0x681ffde0 r8:0x255efd0 r9:0x3 r10:0x2561000 r11:0x246 r12:0x7f9c7380b620 r13:0x7f9c73809438 r14:0x0 r15:0x7f9c73808ba0 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xc fs_base:0x7f9c74b2ab80 gs_base:0x0
  { map_file:"<ZERO>", addr:0x257f000, length:(nil), prot_flags:"---p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x0 }
}
{
  real_time:3649183.888924 global_time:1020, event:`SYSCALLBUF_FLUSH' tid:9596, ticks:3761234
  { syscall:'access', ret:0xfffffffffffffffe, size:0x10 }
  { syscall:'access', ret:0x0, size:0x10 }
}
{
  real_time:3649183.888927 global_time:1021, event:`SYSCALL: sched_yield' (state:ENTERING_SYSCALL) tid:9596, ticks:3761234
rax:0xffffffffffffffda rbx:0x7f9c7380f240 rcx:0xffffffffffffffff rdx:0x7f9c74b3c908 rsi:0x0 rdi:0x7f9c74b3c908 rbp:0x2550150 rsp:0x7ffca763e9c8 r8:0x0 r9:0x7f9c7380e698 r10:0x26 r11:0x246 r12:0x101 r13:0x7ffca763ed90 r14:0x0 r15:0x0 rip:0x7f9c7470900b eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x18 fs_base:0x7f9c74b2ab80 gs_base:0x0
}
{
  real_time:3649183.888929 global_time:1022, event:`SYSCALLBUF_RESET' tid:9596, ticks:3761234
}
{
  real_time:3649183.888946 global_time:1023, event:`SYSCALL: sched_yield' (state:EXITING_SYSCALL) tid:9596, ticks:3761234
rax:0x0 rbx:0x7f9c7380f240 rcx:0xffffffffffffffff rdx:0x7f9c74b3c908 rsi:0x0 rdi:0x7f9c74b3c908 rbp:0x2550150 rsp:0x7ffca763e9c8 r8:0x0 r9:0x7f9c7380e698 r10:0x26 r11:0x246 r12:0x101 r13:0x7ffca763ed90 r14:0x0 r15:0x0 rip:0x7f9c7470900b eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x18 fs_base:0x7f9c74b2ab80 gs_base:0x0
}
{
  real_time:3649183.889035 global_time:1024, event:`SYSCALLBUF_FLUSH' tid:9596, ticks:3762684
  { syscall:'openat', ret:0x3, size:0x10, desched:1 }
  { syscall:'readlinkat', ret:0x15, size:0x25 }
  { syscall:'read', ret:0x340, size:0x350, desched:1 }
  { syscall:'fstat', ret:0x0, size:0xa0 }
}
{
  real_time:3649183.889038 global_time:1025, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:9596, ticks:3762684
rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x5 rsi:0x7e4d28 rdi:0x0 rbp:0x0 rsp:0x681ffde0 r8:0x3 r9:0x0 r10:0x802 r11:0x246 r12:0x5 r13:0x7e4d28 r14:0x802 r15:0x3 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7f9c74b2ab80 gs_base:0x0
}
{
  real_time:3649183.889039 global_time:1026, event:`SYSCALLBUF_RESET' tid:9596, ticks:3762684
}
{
  real_time:3649183.890963 global_time:1027, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:9596, ticks:3762684
rax:0x7f9c6a607000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x5 rsi:0x7e4d28 rdi:0x0 rbp:0x0 rsp:0x681ffde0 r8:0x3 r9:0x0 r10:0x802 r11:0x246 r12:0x5 r13:0x7e4d28 r14:0x802 r15:0x3 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7f9c74b2ab80 gs_base:0x0
  { map_file:"/tmp/test.so", addr:0x7f9c6a607000, length:0x7e5000, prot_flags:"r-xp", file_offset:0x0, device:64768, inode:33567816, data_file:"/tmp/test.so", data_offset:0x0, file_size:0x2abe80 }
}
{
  real_time:3649183.891042 global_time:1028, event:`SYSCALLBUF_FLUSH' tid:9596, ticks:3762727
  { syscall:'mprotect', ret:0x0, size:0x10 }
}
{
  real_time:3649183.891049 global_time:1029, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:9596, ticks:3762727
rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0xa5000 rdi:0x7f9c6a9e6000 rbp:0x7f9c6a9e6000 rsp:0x681ffde0 r8:0x3 r9:0x1df000 r10:0x812 r11:0x246 r12:0x3 r13:0xa5000 r14:0x812 r15:0x3 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7f9c74b2ab80 gs_base:0x0
}
{
  real_time:3649183.891051 global_time:1030, event:`SYSCALLBUF_RESET' tid:9596, ticks:3762727
}
{
  real_time:3649183.891094 global_time:1031, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:9596, ticks:3762727
rax:0x7f9c6a9e6000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0xa5000 rdi:0x7f9c6a9e6000 rbp:0x7f9c6a9e6000 rsp:0x681ffde0 r8:0x3 r9:0x1df000 r10:0x812 r11:0x246 r12:0x3 r13:0xa5000 r14:0x812 r15:0x3 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7f9c74b2ab80 gs_base:0x0
  { map_file:"/tmp/test.so", addr:0x7f9c6a9e6000, length:0xa5000, prot_flags:"rw-p", file_offset:0x1df000, device:64768, inode:33567816, data_file:"/tmp/test.so", data_offset:0x1df000, file_size:0x2abe80 }
}
{
  real_time:3649183.891206 global_time:1032, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:9596, ticks:3762811
rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x360d28 rdi:0x7f9c6aa8b000 rbp:0x7f9c6aa8b000 rsp:0x681ffde0 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x3 r13:0x360d28 r14:0x32 r15:0xffffffff rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7f9c74b2ab80 gs_base:0x0
}
{
  real_time:3649183.891231 global_time:1033, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:9596, ticks:3762811
rax:0x7f9c6aa8b000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x360d28 rdi:0x7f9c6aa8b000 rbp:0x7f9c6aa8b000 rsp:0x681ffde0 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x3 r13:0x360d28 r14:0x32 r15:0xffffffff rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7f9c74b2ab80 gs_base:0x0
  { map_file:"<ZERO>", addr:0x7f9c6aa8b000, length:0x361000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x361000 }
}
{
  real_time:3649183.891394 global_time:1034, event:`SYSCALLBUF_FLUSH' tid:9596, ticks:3771993
  { syscall:'close', ret:0x0, size:0x10 }
  { syscall:'mprotect', ret:0x0, size:0x10 }
}
{
  real_time:3649183.891427 global_time:1035, event:`PATCH_SYSCALL' tid:9596, ticks:3771993
rax:0x6 rbx:0x2560680 rcx:0xffffffffffffffff rdx:0x7ffca763e8c0 rsi:0x7ffca763e8c0 rdi:0x255f680 rbp:0x7ffca763e990 rsp:0x7ffca763e878 r8:0x2550153 r9:0x3 r10:0x0 r11:0x246 r12:0x255f680 r13:0x2550154 r14:0x2550151 r15:0x255f684 rip:0x7f9c73537d87 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f9c74b2ab80 gs_base:0x0
  { tid:9596, addr:0x7f9c7493d773, length:0x4f }
  { tid:9596, addr:0x7f9c73537d87, length:0x5 }
  { tid:9596, addr:0x7f9c73537d8c, length:0x3 }
}
{
  real_time:3649183.891429 global_time:1036, event:`SYSCALLBUF_RESET' tid:9596, ticks:3771993
}
{
  real_time:3649183.893855 global_time:1037, event:`SYSCALLBUF_FLUSH' tid:9596, ticks:3828174
  { syscall:'lstat', ret:0x0, size:0xa0 }
  { syscall:'lstat', ret:0x0, size:0xa0 }
  { syscall:'lstat', ret:0x0, size:0xa0 }
  { syscall:'lstat', ret:0x0, size:0xa0 }
  { syscall:'access', ret:0xfffffffffffffffe, size:0x10 }
  { syscall:'access', ret:0x0, size:0x10 }
}
{
  real_time:3649183.893861 global_time:1038, event:`SYSCALL: sched_yield' (state:ENTERING_SYSCALL) tid:9596, ticks:3828174
rax:0xffffffffffffffda rbx:0x7f9c7380f240 rcx:0xffffffffffffffff rdx:0x7f9c74b3c908 rsi:0x0 rdi:0x7f9c74b3c908 rbp:0x2550150 rsp:0x7ffca763d468 r8:0x0 r9:0x7f9c7380e698 r10:0x27 r11:0x246 r12:0x101 r13:0x7ffca763ed90 r14:0x0 r15:0x0 rip:0x7f9c7470900b eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x18 fs_base:0x7f9c74b2ab80 gs_base:0x0
}
=== Start rr backtrace:
rr(_ZN2rr13dump_rr_stackEv+0x5a)[0x55ef8181ffea]
rr(_ZN2rr9GdbServer15emergency_debugEPNS_4TaskE+0x645)[0x55ef8190bce5]
rr(+0x128e70)[0x55ef818f8e70]
rr(_ZN2rr21EmergencyDebugOstreamD1Ev+0x73)[0x55ef818f8fd3]
rr(_ZN2rr13ReplaySession16flush_syscallbufEPNS_10ReplayTaskERKNS0_15StepConstraintsE+0xc24)[0x55ef81880f24]
rr(_ZN2rr13ReplaySession18try_one_trace_stepEPNS_10ReplayTaskERKNS0_15StepConstraintsE+0x18e)[0x55ef8188569e]
rr(_ZN2rr13ReplaySession11replay_stepERKNS0_15StepConstraintsE+0x134)[0x55ef81886054]
rr(_ZN2rr14ReplayTimeline19replay_step_forwardENS_10RunCommandEl+0xd2)[0x55ef81864002]
rr(_ZN2rr9GdbServer14debug_one_stepERNS_10GdbRequestE+0xb66)[0x55ef8190d9e6]
rr(_ZN2rr9GdbServer12serve_replayERKNS0_15ConnectionFlagsE+0x72b)[0x55ef8190e53b]
rr(+0xbf9be)[0x55ef8188f9be]
rr(_ZN2rr13ReplayCommand3runERSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x4f3)[0x55ef818901d3]
rr(main+0x2a2)[0x55ef81811032]
/lib64/libc.so.6(__libc_start_main+0xf3)[0x7f5f2b7c27b3]
rr(_start+0x2e)[0x55ef8181118e]
=== End rr backtrace

    Produced by a run of

    rr replay
(gdb) break myfunc_
(gdb) continue
(gdb) step

    If I do the same on gdb --args (adding run after the break) I just step into the function called (which is in a library already loaded).

    opened by GitMensch 53
  • Memory contents diverging

    Memory contents diverging

    Somewhat like #416, except that

    1. Only reproduces with syscall buffering enabled
    2. Only reproduces 1/10 or 1/20 runs or so
    3. Reproduces at what looks like fairly arbitrary points in execution, as opposed to relatively early in startup
    4. Overshoot is by a widely variable amount; I've seen from ~50 to ~2400 rcbs

    Smells to me like a memory divergence, but I don't have any specific guesses yet. Reproduces with

    rr -v record -b "../ff-prof/dist/bin/firefox" -no-remote -P garbage -reftest file:///home/cjones/rr/mozilla-central/layout/reftests/transform/reftest.list

    but doesn't seem to reproduce with the (quick) network reftests used for #416.

    opened by joneschrisg 53
  • Failing tests under RHEL 8.1 QEMU/KVM guest

    Failing tests under RHEL 8.1 QEMU/KVM guest

    I have the following setup:

    • QEMU/KVM guest: Red Hat Enterprise Linux 8.1 (x86_64)
    • KVM host: Red Hat Enterprise Linux 8.1 (x86_64)
    • Git repository last updated May13th, 2020

    When I run cmake --build . --target test, 38 out of 2423 tests fail. I have attached the test output to this report.

    Please let me know if there is any further information I can provide. Thanks.

    TestOutput.txt

    opened by kavinda-forcepoint 52
  • Provide an API to create tracee-shared memory mappings

    Provide an API to create tracee-shared memory mappings

    @rocallahan as discussed on IRC, this generalizes the syscallbuf code to work for other mappings that are shared with the tracee as well and provides an API to replace an existing private mapping with one that is shared with the tracer. The intention of this change is to be able to speed up memory operations on a given segment. This is not quite done yet, but I wanted to put the code up to get some early feedback (or feel free to wait until it's done - that would be fine as well).

    cc @carnaval

    opened by Keno 52
  • Crash on replay from within gdb

    Crash on replay from within gdb

    Using rr 5.2.0 from Ubuntu 18.04.2 LTS.

    I am recording a successful run of an application with 'rr record X'. When I replay with 'rr replay' and then 'continue' from within gdb, I am getting a crash within a proprietary library for which I have neither source nor debug information.

    I am not seeing the crash when I either replay with 'rr replay -a' or record with 'rr record -n'. In both cases, the replay is true to the original recording.

    Given the proprietary nature of said library, I'm willing to fix the issue on my machine but I'd be glad for hints on where to start.

    opened by bkin 49
  • Improving thread creation performance

    Improving thread creation performance

    I'm looking at a large application which creates absurd numbers of threads during startup. I'd like to reduce the overhead of this code under rr. Measuring the thread_stress test suggests that creating a thread that does practically nothing but exit takes about 5ms under rr on my machine.

    The key problem seems to be the work we have to do to get syscallbuf set up for each thread. This is basically:

    • 2 round trips to rr for gettid enter/exit (syscallbuf)
    • 2 round trips to rr for perf_event_open enter/exit (syscallbuf)
    • 2 round trips to rr for fcntl dup enter/exit (syscallbuf)
    • 2 round trips to rr for rrcall_init_buffers (syscallbuf)
      • 3 round trips to tracee (recvmsg, mmap, close) for Session::create_shared_mmap
      • 4 round trips to tracee (sendmsg desched fd, recvmsg/dup3/close clonedata fd) for RecordTask::init_buffers_arch

    That's 15 round trips, i.e. 30 context switches, in code we control.

    opened by rocallahan 3
  • Early Draft: Add parameter to check if files are mapped in processes not part of the recording.

    Early Draft: Add parameter to check if files are mapped in processes not part of the recording.

    In its current state it just calls lsof with the pids inside the the recording excluded.

    It would show the mapping that caused the issue in #3417, but the others are probably just read-only mappings due to /proc/pid/maps.

    $ rr record --lsof -n atril sshfs/home/benutzer/test2.pdfrr: Saving execution to trace directory `/home/bernhard/data/entwicklung/2022/rr/rr-recordings/atril-14'.
Warning: /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache seems to be mapped by a process outside of the recording. Diversions might happen when replaying this trace.
Warning: /home/bernhard/.cache/fontconfig/a6050266-6adb-4e1c-8a1d-f55d18a3f9c3-le64.cache-7 seems to be mapped by a process outside of the recording. Diversions might happen when replaying this trace.
Warning: /run/user/1000/dconf/user seems to be mapped by a process outside of the recording. Diversions might happen when replaying this trace.
Warning: /home/bernhard/.local/share/mime/mime.cache seems to be mapped by a process outside of the recording. Diversions might happen when replaying this trace.
Warning: /usr/local/share/mime/mime.cache seems to be mapped by a process outside of the recording. Diversions might happen when replaying this trace.
Warning: /usr/share/mime/mime.cache seems to be mapped by a process outside of the recording. Diversions might happen when replaying this trace.
Warning: /home/bernhard/.local/share/gvfs-metadata/root seems to be mapped by a process outside of the recording. Diversions might happen when replaying this trace.
Warning: /home/bernhard/.local/share/gvfs-metadata/root-5c301c7a.log seems to be mapped by a process outside of the recording. Diversions might happen when replaying this trace.
...
    opened by bernhardu 0
  • Something in the 6.0.11 -> 6.0.12 kernel update breaks rr

    Something in the 6.0.11 -> 6.0.12 kernel update breaks rr

    When I update my kernel to 6.0.12.arch1-1, rr stops working (replay fails with counting ticks assertions). I was on an old rev, but reproduces on latest master as well.

    This is on a Zen machine. lscpu (on 6.0.11):

    $ lscpu                                                                                                                                                                                                                                   
Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         43 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  128
  On-line CPU(s) list:   0-127
Vendor ID:               AuthenticAMD
  Model name:            AMD Ryzen Threadripper 3990X 64-Core Processor
    CPU family:          23
    Model:               49
    Thread(s) per core:  2
    Core(s) per socket:  64
    Socket(s):           1
    Stepping:            0
    Frequency boost:     enabled
    CPU(s) scaling MHz:  77%
    CPU max MHz:         2900.0000
    CPU min MHz:         2200.0000
    BogoMIPS:            5802.69
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmper
                         f rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfc
                         tr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm
                         _llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsav
                         e_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca sev sev_es
Virtualization features: 
  Virtualization:        AMD-V
Caches (sum of all):     
  L1d:                   2 MiB (64 instances)
  L1i:                   2 MiB (64 instances)
  L2:                    32 MiB (64 instances)
  L3:                    256 MiB (16 instances)
NUMA:                    
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-127
Vulnerabilities:         
  Itlb multihit:         Not affected
  L1tf:                  Not affected
  Mds:                   Not affected
  Meltdown:              Not affected
  Mmio stale data:       Not affected
  Retbleed:              Mitigation; untrained return thunk; SMT enabled with STIBP protection
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; Retpolines, IBPB conditional, STIBP always-on, RSB filling, PBRSB-eIBRS Not affected
  Srbds:                 Not affected
  Tsx async abort:       Not affected

    If someone gives me a few pointers on how, I'm happy to try bisecting or what not. I'd be curious to know if this is CPU-specific or other people can repro as well.

    opened by emilio 8
  • Persistent Checkpointing PR (#2184)

    Persistent Checkpointing PR (#2184)

    This is the first version of a PR that attempts to provide the functionality requested by issue #2184

    • rr create-checkpoints -i <some_interval> [-s <some_start_event>, -e <some_end_event>] where last two params are optional.
    • rr replay -g <evt> spawns the session from the most recent PCP before <evt>. It also spaws the most recent PCP, if -f <pid> is used, i.e. it finds when <pid> is created and spawns first PCP before that.
    • rr replay uses PCP during reverse execution
    • rr rerun uses PCP as well

    Both the replay and rerun command now takes --ignore-pcp to ignore any PCP's and I've made spawning from PCP the default behavior of both commands.

    2 commands has also been added to the spawned GDB; write-checkpoints and load-checkpoints.

    The last point about persistent checkpoints being created at record time is not provided by this PR, but I'm willing to attempt to add that in a future PR, now that I have a little insight into how this would/could/maybe should work.

    At this time, little to no optimizations are performed. Each mapping in the process address space is serialized to disk and it is currently not compressed in any way. Compressing data that goes into anonymous mappings should be fairly simple to implement as this data will get copied into memory during restore of a PCP, while file backed mappings (like executable data for instance) can not be compressed as easily. One wants to map as much file backed as possible, as this is not necessarily committed to physical memory immediately, which is the case with copying data into mappings.

    Other optimizations that possibly could be done, is to instead of creating each checkpoints "from scratch", is to during restore of PCP's, reconstitute the first one (at event N), then when reconstituting the following checkpoint, fork the first and make the changes required to that one. As it stands right now, it creates a new session for each checkpoint. Theoretically this consumes more memory. Forking checkpoint N+1 from N and changing the address space where needed, I think would mean that less memory is used, I think.

    Also, if anybody has any ideas on how one could possibly write tests for something like this, they would be most welcome to share those thoughts with me.

    opened by theIDinside 0
  • Add Intel Tremont

    Add Intel Tremont

    The cpu_type is from an Intel N6005 [1].

    [1] https://www.intel.com/content/www/us/en/products/sku/212327/intel-pentium-silver-n6005-processor-4m-cache-up-to-3-30-ghz/specifications.html

    Recently got my hands on a new board with this chip. Since it's a successor of Goldmont Plus, I gave the same configuration a shot and it looked ok(?)

    Failed tests:

    The following tests FAILED:
	468 - prctl (Failed)
	469 - prctl-no-syscallbuf (Failed)
	724 - sioc (Failed)
	725 - sioc-no-syscallbuf (Failed)
	1140 - x86/string_instructions_replay (Failed)
	1141 - x86/string_instructions_replay-no-syscallbuf (Failed)
	1144 - x86/string_instructions_watch (Failed)
	1145 - x86/string_instructions_watch-no-syscallbuf (Failed)
	1388 - x86/string_instructions_break (Failed)
	1389 - x86/string_instructions_break-no-syscallbuf (Failed)
	1390 - x86/string_instructions_replay_quirk (Failed)
	1391 - x86/string_instructions_replay_quirk-no-syscallbuf (Failed)
	1892 - prctl-32 (Failed)
	1893 - prctl-32-no-syscallbuf (Failed)
	2564 - x86/string_instructions_replay-32 (Failed)
	2565 - x86/string_instructions_replay-32-no-syscallbuf (Failed)
	2568 - x86/string_instructions_watch-32 (Failed)
	2569 - x86/string_instructions_watch-32-no-syscallbuf (Failed)
	2812 - x86/string_instructions_break-32 (Failed)
	2813 - x86/string_instructions_break-32-no-syscallbuf (Failed)
	2814 - x86/string_instructions_replay_quirk-32 (Failed)
	2815 - x86/string_instructions_replay_quirk-32-no-syscallbuf (Failed)

    Full log: rr_check.log

    opened by Frederick888 2
  • PerfCounters: Add support for AMD Family 15h Model 2 (Piledriver)

    PerfCounters: Add support for AMD Family 15h Model 2 (Piledriver)

    Extends existing Family 15h Model 30 (Steamroller) support for Piledriver. Piledriver supports PMCx0C4 (Retired Taken Branch Instructions) and PMCx0C6 (Retired Far Control Transfer), just like Model 30h. 1

    Note that PMCx0C4 counts all control flow changes, including exceptions and interrupts. AFAICT on 15h, there is no PMC for just retired conditional branches.

    Tested:

    1. counters-test: vsrinivas@ubuntu:~/tmp/rr/src/counters-test$ cc -O2 counters.c vsrinivas@ubuntu:~/tmp/rr/src/counters-test$ sudo ./a.out Ticks mismatch; got 1003, expected 1002 Aborted

      Varying the number of volatile matches, we always see one more tick than expected, which I think is a RET instruction.

    2. ctest: 97% tests passed, 45 tests failed out of 1425

    Total Test time (real) = 3092.96 sec

    The following tests FAILED: 53 - x86/chew_cpu_cpuid-no-syscallbuf (Failed) 110 - detach_state (Failed) 162 - x86/fault_in_code_page (Failed) 558 - x86/rdtsc_flags (Failed) 724 - sioc (Failed) 725 - sioc-no-syscallbuf (Failed) 842 - vsyscall (Failed) 843 - vsyscall-no-syscallbuf (Failed) 844 - vsyscall_timeslice (Failed) 845 - vsyscall_timeslice-no-syscallbuf (Failed) 846 - x86/x87env (Failed) 847 - x86/x87env-no-syscallbuf (Failed) 888 - async_signal_syscalls (Failed) 890 - async_signal_syscalls2 (Failed) 910 - x86/blocked_sigsegv (Failed) 916 - breakpoint_overlap (Failed) 924 - checkpoint_dying_threads (Failed) 932 - clone_interruption (Failed) 938 - conditional_breakpoint_offload (Failed) 939 - conditional_breakpoint_offload-no-syscallbuf (Failed) 951 - daemon_read-no-syscallbuf (Failed) 962 - dlopen (Failed) 980 - exit_race (Failed) 981 - exit_race-no-syscallbuf (Failed) 984 - x86/explicit_checkpoints (Failed) 1072 - x86/rdtsc_loop (Failed) 1080 - reverse_continue_breakpoint (Failed) 1081 - reverse_continue_breakpoint-no-syscallbuf (Failed) 1089 - reverse_step_long-no-syscallbuf (Failed) 1092 - reverse_step_threads_break (Failed) 1100 - rseq_syscallbuf (Failed) 1130 - strict_priorities (Failed) 1131 - strict_priorities-no-syscallbuf (Failed) 1150 - x86/syscallbuf_rdtsc_page (Failed) 1174 - thread_open_race (Failed) 1206 - watchpoint_at_sched (Failed) 1208 - watchpoint_before_signal (Failed) 1209 - watchpoint_before_signal-no-syscallbuf (Failed) 1218 - async_signal_syscalls_100 (Failed) 1219 - async_signal_syscalls_100-no-syscallbuf (Failed) 1320 - record_replay (Failed) 1321 - record_replay-no-syscallbuf (Failed) 1354 - reverse_watchpoint_syscall (Failed) 1418 - vsyscall_singlestep (Failed) 1419 - vsyscall_singlestep-no-syscallbuf (Failed)

    opened by vsrinivas 2
Releases(5.6.0)
Owner
null
GitHub http://rr-project.org/
Log agent to replay time-stamped log stream

replayman Log agent to replay time-stamped log stream. Getting started Installation cargo install replayman Prepare for your data TSV with a header li

SpringQL 3 Dec 23, 2022
Semi-automatic OSINT framework and package manager

sn0int sn0int (pronounced /snoɪnt/) is a semi-automatic OSINT framework and package manager. It was built for IT security professionals and bug hunter

null 1.4k Dec 31, 2022
link is a command and control framework written in rust

link link is a command and control framework written in rust. Currently in alpha. Table of Contents Introduction Features Feedback Build Process Ackno

null 427 Dec 24, 2022
Rust implementation of The Update Framework (TUF)

rust-tuf A Rust implementation of The Update Framework (TUF). Full documentation is hosted at docs.rs. Warning: Beta Software This is under active dev

heartsucker 152 Dec 11, 2022
A fuzzer framework built in Rust

lain This crate provides functionality one may find useful while developing a fuzzer. A recent nightly Rust build is required for the specialization f

Microsoft 469 Dec 9, 2022
Binary Analysis Framework in Rust

Welcome to Falcon Falcon is a formal binary analysis framework in Rust. Expression-based IL with strong influences from RREIL and Binary Ninja's LLIL.

Falcon Binary Analysis Framework 489 Dec 18, 2022
Bindings to the macOS Security.framework

macOS/iOS Security framework for Rust Documentation Bindings to the Apple's Security.framework. Allows use of TLS and Keychain from Rust. License Lice

Kornel 172 Jan 2, 2023
Kepler is a vulnerability database and lookup store and API currently utilising National Vulnerability Database and NPM Advisories as data sources

Kepler — Kepler is a vulnerability database and lookup store and API currently utilising National Vulnerability Database and NPM Advisories as data so

Exein.io 101 Nov 12, 2022
Steals browser passwords and cookies and sends to webhook.

Browser-Stealer Steals browser passwords and cookies and sends to webhook. Donating Educational Purposes Only This code is made so you can learn from

RadonCoding 3 Sep 27, 2021
Xori is an automation-ready disassembly and static analysis library for PE32, 32+ and shellcode

Xori - Custom disassembly framework Xori is an automation-ready disassembly and static analysis library that consumes shellcode or PE binaries and pro

ENDGAME 712 Nov 28, 2022
🕵️‍♀️ Find, locate, and query files for ops and security experts ⚡️⚡️⚡️

Recon Find, locate, and query files for ops and security experts Key Features • How To Use • Download • Contributing • License Key Features Query with

Rusty Ferris Club 11 Dec 16, 2022
A Comprehensive Web Fuzzer and Content Discovery Tool

rustbuster A Comprehensive Web Fuzzer and Content Discovery Tool Introduction Check the blog post: Introducing Rustbuster — A Comprehensive Web Fuzzer

Francesco Soncina 467 Dec 26, 2022
A simple menu to keep all your most used one-liners and scripts in one place

Dama Desktop Agnostic Menu Aggregate This program aims to be a hackable, easy to use menu that can be paired to lightweight window managers in order t

null 47 Jul 23, 2022
A simple scanner that loops through ips and checks if a minecraft server is running on port 25565

scanolotl Scanolotl is a simple scanner that loops through ips and checks if a minecraft server is running on port 25565. Scanolotl can also preform a

JustFr33z 3 Jul 28, 2022
Rust library for building and running BPF/eBPF modules

RedBPF A Rust eBPF toolchain. Overview The redbpf project is a collection of tools and libraries to build eBPF programs using Rust. It includes: redbp

foniod 1.5k Jan 1, 2023
telemetry aggregation and shipping, last up the ladder

cernan - telemetry aggregation and shipping, last up the ladder Cernan is a telemetry and logging aggregation server. It exposes multiple interfaces f

Postmates Inc. 311 Nov 21, 2022
Automated attack surface mapper and vulnerability scanner

Phaser Automated attack surface mapper and vulnerability scanner What is this? Phaser is a high-performance and automated attack surface mapper and vu

Sylvain Kerkour 74 Dec 16, 2022
unfuck is a utility and library for deobfuscating obfuscated Python 2.7 bytecode

unfuck is a utility and library for deobfuscating obfuscated Python 2.7 bytecode. It is essentially a reimplementation of the Python VM with taint tracking.

Lander Brandt 171 Dec 14, 2022
Finds matching solidity function signatures for a given 4 byte signature hash and arguments.

Finds matching solidity function signatures for a given 4 byte signature hash and arguments. Useful for finding collisions or 0x00000000 gas saving methods (though there are better techniques for saving gas on calldata)

null 73 Dec 22, 2022