Buildresult: linus/sun3_defconfig/m68k-gcc8 built on Apr 7 2019, 04:47

kisskb

Revisions | Branches | Compilers | Configs | Build Results | Build Failures |

Status:	OK
Date/Time:	Apr 7 2019, 04:47
Duration:	0:01:09.251417
Builder:	ka1
Revision:	fs: stream_open - opener for stream-like files so that read and write can run simultaneously without deadlock (`10dce8af34226d90fa56746a934f8da5dcdba3df)`
Target:	linus/sun3_defconfig/m68k-gcc8
Branch:	linus
Compiler:	m68k-gcc8 (m68k-linux-gcc (GCC) 4.6.3 / GNU ld (GNU Binutils) 2.22)
Config:	sun3_defconfig (download)
Log:	Download original Possible warnings (36) arch/m68k/configs/sun3_defconfig:306:warning: symbol value 'm' invalid for NET_DEVLINK arch/m68k/configs/sun3_defconfig:423:warning: symbol value 'm' invalid for FS_ENCRYPTION WARNING: unmet direct dependencies detected for NEED_MULTIPLE_NODES WARNING: unmet direct dependencies detected for NEED_MULTIPLE_NODES WARNING: unmet direct dependencies detected for NEED_MULTIPLE_NODES <stdin>:1478:2: warning: #warning syscall pidfd_send_signal not implemented [-Wcpp] <stdin>:1481:2: warning: #warning syscall io_uring_setup not implemented [-Wcpp] <stdin>:1484:2: warning: #warning syscall io_uring_enter not implemented [-Wcpp] <stdin>:1487:2: warning: #warning syscall io_uring_register not implemented [-Wcpp] kernel/printk/printk.c:187:16: warning: 'old' may be used uninitialized in this function [-Wuninitialized] ipc/shm.c:1712:59: warning: 'file' may be used uninitialized in this function [-Wuninitialized] kernel/acct.c:177:2: warning: value computed is not used [-Wunused-value] lib/mpi/mpicoder.c:336:12: warning: 'buff' may be used uninitialized in this function [-Wuninitialized] mm/mprotect.c:42:20: warning: unused variable 'mm' [-Wunused-variable] net/core/gen_stats.c:161:19: warning: 'seq' may be used uninitialized in this function [-Wuninitialized] include/linux/list.h:65:12: warning: 'pdeo' may be used uninitialized in this function [-Wuninitialized] lib/rhashtable.c:264:2: warning: 'next' may be used uninitialized in this function [-Wuninitialized] net/core/dev.c:3405:15: warning: 'tail' may be used uninitialized in this function [-Wuninitialized] net/core/filter.c:3479:4: warning: value computed is not used [-Wunused-value] fs/udf/unicode.c:132:8: warning: 'c' may be used uninitialized in this function [-Wuninitialized] fs/posix_acl.c:147:3: warning: value computed is not used [-Wunused-value] fs/ocfs2/alloc.c:7604:17: warning: 'first_bit' may be used uninitialized in this function [-Wuninitialized] drivers/net/ethernet/i825xx/sun3_82586.c:993:89: warning: array subscript is above array bounds [-Warray-bounds] fs/ocfs2/file.c:2386:3: warning: value computed is not used [-Wunused-value] include/linux/skbuff.h:1397:6: warning: 'extra_uref' may be used uninitialized in this function [-Wuninitialized] include/linux/skbuff.h:1397:6: warning: 'extra_uref' may be used uninitialized in this function [-Wuninitialized] include/net/sock.h:1949:19: warning: 'sk' may be used uninitialized in this function [-Wuninitialized] drivers/net/macsec.c:1926:2: warning: 'assoc_num' may be used uninitialized in this function [-Wuninitialized] drivers/net/macsec.c:1926:2: warning: 'rx_sc' may be used uninitialized in this function [-Wuninitialized] drivers/net/macsec.c:2003:2: warning: 'assoc_num' may be used uninitialized in this function [-Wuninitialized] drivers/net/macsec.c:2003:2: warning: 'tx_sc' may be used uninitialized in this function [-Wuninitialized] drivers/net/macsec.c:2068:5: warning: 'assoc_num' may be used uninitialized in this function [-Wuninitialized] drivers/net/macsec.c:2068:24: warning: 'tx_sc' may be used uninitialized in this function [-Wuninitialized] drivers/net/macsec.c:2069:21: warning: 'secy' may be used uninitialized in this function [-Wuninitialized] drivers/net/tun.c:1842:30: warning: 'copylen' may be used uninitialized in this function [-Wuninitialized] drivers/net/tun.c:1535:31: warning: 'linear' may be used uninitialized in this function [-Wuninitialized] Full Log # git rev-parse -q --verify 10dce8af34226d90fa56746a934f8da5dcdba3df^{commit} 10dce8af34226d90fa56746a934f8da5dcdba3df already have revision, skipping fetch # git checkout -q -f -B kisskb 10dce8af34226d90fa56746a934f8da5dcdba3df # git clean -qxdf # < git log -1 # commit 10dce8af34226d90fa56746a934f8da5dcdba3df # Author: Kirill Smelkov <kirr@nexedi.com> # Date: Tue Mar 26 22:20:43 2019 +0000 # # fs: stream_open - opener for stream-like files so that read and write can run simultaneously without deadlock # # Commit 9c225f2655e3 ("vfs: atomic f_pos accesses as per POSIX") added # locking for file.f_pos access and in particular made concurrent read and # write not possible - now both those functions take f_pos lock for the # whole run, and so if e.g. a read is blocked waiting for data, write will # deadlock waiting for that read to complete. # # This caused regression for stream-like files where previously read and # write could run simultaneously, but after that patch could not do so # anymore. See e.g. commit 581d21a2d02a ("xenbus: fix deadlock on writes # to /proc/xen/xenbus") which fixes such regression for particular case of # /proc/xen/xenbus. # # The patch that added f_pos lock in 2014 did so to guarantee POSIX thread # safety for read/write/lseek and added the locking to file descriptors of # all regular files. In 2014 that thread-safety problem was not new as it # was already discussed earlier in 2006. # # However even though 2006'th version of Linus's patch was adding f_pos # locking "only for files that are marked seekable with FMODE_LSEEK (thus # avoiding the stream-like objects like pipes and sockets)", the 2014 # version - the one that actually made it into the tree as 9c225f2655e3 - # is doing so irregardless of whether a file is seekable or not. # # See # # https://lore.kernel.org/lkml/53022DB1.4070805@gmail.com/ # https://lwn.net/Articles/180387 # https://lwn.net/Articles/180396 # # for historic context. # # The reason that it did so is, probably, that there are many files that # are marked non-seekable, but e.g. their read implementation actually # depends on knowing current position to correctly handle the read. Some # examples: # # kernel/power/user.c snapshot_read # fs/debugfs/file.c u32_array_read # fs/fuse/control.c fuse_conn_waiting_read + ... # drivers/hwmon/asus_atk0110.c atk_debugfs_ggrp_read # arch/s390/hypfs/inode.c hypfs_read_iter # ... # # Despite that, many nonseekable_open users implement read and write with # pure stream semantics - they don't depend on passed ppos at all. And for # those cases where read could wait for something inside, it creates a # situation similar to xenbus - the write could be never made to go until # read is done, and read is waiting for some, potentially external, event, # for potentially unbounded time -> deadlock. # # Besides xenbus, there are 14 such places in the kernel that I've found # with semantic patch (see below): # # drivers/xen/evtchn.c:667:8-24: ERROR: evtchn_fops: .read() can deadlock .write() # drivers/isdn/capi/capi.c:963:8-24: ERROR: capi_fops: .read() can deadlock .write() # drivers/input/evdev.c:527:1-17: ERROR: evdev_fops: .read() can deadlock .write() # drivers/char/pcmcia/cm4000_cs.c:1685:7-23: ERROR: cm4000_fops: .read() can deadlock .write() # net/rfkill/core.c:1146:8-24: ERROR: rfkill_fops: .read() can deadlock .write() # drivers/s390/char/fs3270.c:488:1-17: ERROR: fs3270_fops: .read() can deadlock .write() # drivers/usb/misc/ldusb.c:310:1-17: ERROR: ld_usb_fops: .read() can deadlock .write() # drivers/hid/uhid.c:635:1-17: ERROR: uhid_fops: .read() can deadlock .write() # net/batman-adv/icmp_socket.c:80:1-17: ERROR: batadv_fops: .read() can deadlock .write() # drivers/media/rc/lirc_dev.c:198:1-17: ERROR: lirc_fops: .read() can deadlock .write() # drivers/leds/uleds.c:77:1-17: ERROR: uleds_fops: .read() can deadlock .write() # drivers/input/misc/uinput.c:400:1-17: ERROR: uinput_fops: .read() can deadlock .write() # drivers/infiniband/core/user_mad.c:985:7-23: ERROR: umad_fops: .read() can deadlock .write() # drivers/gnss/core.c:45:1-17: ERROR: gnss_fops: .read() can deadlock .write() # # In addition to the cases above another regression caused by f_pos # locking is that now FUSE filesystems that implement open with # FOPEN_NONSEEKABLE flag, can no longer implement bidirectional # stream-like files - for the same reason as above e.g. read can deadlock # write locking on file.f_pos in the kernel. # # FUSE's FOPEN_NONSEEKABLE was added in 2008 in a7c1b990f715 ("fuse: # implement nonseekable open") to support OSSPD. OSSPD implements /dev/dsp # in userspace with FOPEN_NONSEEKABLE flag, with corresponding read and # write routines not depending on current position at all, and with both # read and write being potentially blocking operations: # # See # # https://github.com/libfuse/osspd # https://lwn.net/Articles/308445 # # https://github.com/libfuse/osspd/blob/14a9cff0/osspd.c#L1406 # https://github.com/libfuse/osspd/blob/14a9cff0/osspd.c#L1438-L1477 # https://github.com/libfuse/osspd/blob/14a9cff0/osspd.c#L1479-L1510 # # Corresponding libfuse example/test also describes FOPEN_NONSEEKABLE as # "somewhat pipe-like files ..." with read handler not using offset. # However that test implements only read without write and cannot exercise # the deadlock scenario: # # https://github.com/libfuse/libfuse/blob/fuse-3.4.2-3-ga1bff7d/example/poll.c#L124-L131 # https://github.com/libfuse/libfuse/blob/fuse-3.4.2-3-ga1bff7d/example/poll.c#L146-L163 # https://github.com/libfuse/libfuse/blob/fuse-3.4.2-3-ga1bff7d/example/poll.c#L209-L216 # # I've actually hit the read vs write deadlock for real while implementing # my FUSE filesystem where there is /head/watch file, for which open # creates separate bidirectional socket-like stream in between filesystem # and its user with both read and write being later performed # simultaneously. And there it is semantically not easy to split the # stream into two separate read-only and write-only channels: # # https://lab.nexedi.com/kirr/wendelin.core/blob/f13aa600/wcfs/wcfs.go#L88-169 # # Let's fix this regression. The plan is: # # 1. We can't change nonseekable_open to include &~FMODE_ATOMIC_POS - # doing so would break many in-kernel nonseekable_open users which # actually use ppos in read/write handlers. # # 2. Add stream_open() to kernel to open stream-like non-seekable file # descriptors. Read and write on such file descriptors would never use # nor change ppos. And with that property on stream-like files read and # write will be running without taking f_pos lock - i.e. read and write # could be running simultaneously. # # 3. With semantic patch search and convert to stream_open all in-kernel # nonseekable_open users for which read and write actually do not # depend on ppos and where there is no other methods in file_operations # which assume @offset access. # # 4. Add FOPEN_STREAM to fs/fuse/ and open in-kernel file-descriptors via # steam_open if that bit is present in filesystem open reply. # # It was tempting to change fs/fuse/ open handler to use stream_open # instead of nonseekable_open on just FOPEN_NONSEEKABLE flags, but # grepping through Debian codesearch shows users of FOPEN_NONSEEKABLE, # and in particular GVFS which actually uses offset in its read and # write handlers # # https://codesearch.debian.net/search?q=-%3Enonseekable+%3D # https://gitlab.gnome.org/GNOME/gvfs/blob/1.40.0-6-gcbc54396/client/gvfsfusedaemon.c#L1080 # https://gitlab.gnome.org/GNOME/gvfs/blob/1.40.0-6-gcbc54396/client/gvfsfusedaemon.c#L1247-1346 # https://gitlab.gnome.org/GNOME/gvfs/blob/1.40.0-6-gcbc54396/client/gvfsfusedaemon.c#L1399-1481 # # so if we would do such a change it will break a real user. # # 5. Add stream_open and FOPEN_STREAM handling to stable kernels starting # from v3.14+ (the kernel where 9c225f2655 first appeared). # # This will allow to patch OSSPD and other FUSE filesystems that # provide stream-like files to return FOPEN_STREAM \| FOPEN_NONSEEKABLE # in their open handler and this way avoid the deadlock on all kernel # versions. This should work because fs/fuse/ ignores unknown open # flags returned from a filesystem and so passing FOPEN_STREAM to a # kernel that is not aware of this flag cannot hurt. In turn the kernel # that is not aware of FOPEN_STREAM will be < v3.14 where just # FOPEN_NONSEEKABLE is sufficient to implement streams without read vs # write deadlock. # # This patch adds stream_open, converts /proc/xen/xenbus to it and adds # semantic patch to automatically locate in-kernel places that are either # required to be converted due to read vs write deadlock, or that are just # safe to be converted because read and write do not use ppos and there # are no other funky methods in file_operations. # # Regarding semantic patch I've verified each generated change manually - # that it is correct to convert - and each other nonseekable_open instance # left - that it is either not correct to convert there, or that it is not # converted due to current stream_open.cocci limitations. # # The script also does not convert files that should be valid to convert, # but that currently have .llseek = noop_llseek or generic_file_llseek for # unknown reason despite file being opened with nonseekable_open (e.g. # drivers/input/mousedev.c) # # Cc: Michael Kerrisk <mtk.manpages@gmail.com> # Cc: Yongzhi Pan <panyongzhi@gmail.com> # Cc: Jonathan Corbet <corbet@lwn.net> # Cc: David Vrabel <david.vrabel@citrix.com> # Cc: Juergen Gross <jgross@suse.com> # Cc: Miklos Szeredi <miklos@szeredi.hu> # Cc: Tejun Heo <tj@kernel.org> # Cc: Kirill Tkhai <ktkhai@virtuozzo.com> # Cc: Arnd Bergmann <arnd@arndb.de> # Cc: Christoph Hellwig <hch@lst.de> # Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> # Cc: Julia Lawall <Julia.Lawall@lip6.fr> # Cc: Nikolaus Rath <Nikolaus@rath.org> # Cc: Han-Wen Nienhuys <hanwen@google.com> # Signed-off-by: Kirill Smelkov <kirr@nexedi.com> # Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> # < /opt/cross/kisskb/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux-gcc --version # < /opt/cross/kisskb/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux-ld --version # < git log --format=%s --max-count=1 10dce8af34226d90fa56746a934f8da5dcdba3df # < make -s -j 48 ARCH=m68k O=/kisskb/build/linus_sun3_defconfig_m68k CROSS_COMPILE=/opt/cross/kisskb/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux- sun3_defconfig arch/m68k/configs/sun3_defconfig:306:warning: symbol value 'm' invalid for NET_DEVLINK arch/m68k/configs/sun3_defconfig:423:warning: symbol value 'm' invalid for FS_ENCRYPTION WARNING: unmet direct dependencies detected for NEED_MULTIPLE_NODES Depends on [n]: DISCONTIGMEM [=n] \|\| NUMA Selected by [y]: - SINGLE_MEMORY_CHUNK [=y] && MMU [=y] WARNING: unmet direct dependencies detected for NEED_MULTIPLE_NODES Depends on [n]: DISCONTIGMEM [=n] \|\| NUMA Selected by [y]: - SINGLE_MEMORY_CHUNK [=y] && MMU [=y] WARNING: unmet direct dependencies detected for NEED_MULTIPLE_NODES Depends on [n]: DISCONTIGMEM [=n] \|\| NUMA Selected by [y]: - SINGLE_MEMORY_CHUNK [=y] && MMU [=y] # make -s -j 48 ARCH=m68k O=/kisskb/build/linus_sun3_defconfig_m68k CROSS_COMPILE=/opt/cross/kisskb/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux- <stdin>:1478:2: warning: #warning syscall pidfd_send_signal not implemented [-Wcpp] <stdin>:1481:2: warning: #warning syscall io_uring_setup not implemented [-Wcpp] <stdin>:1484:2: warning: #warning syscall io_uring_enter not implemented [-Wcpp] <stdin>:1487:2: warning: #warning syscall io_uring_register not implemented [-Wcpp] /kisskb/src/kernel/printk/printk.c: In function 'devkmsg_sysctl_set_loglvl': /kisskb/src/kernel/printk/printk.c:187:16: warning: 'old' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/ipc/shm.c: In function 'ksys_shmdt': /kisskb/src/ipc/shm.c:1712:59: warning: 'file' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/kernel/acct.c: In function 'acct_pin_kill': /kisskb/src/kernel/acct.c:177:2: warning: value computed is not used [-Wunused-value] /kisskb/src/lib/mpi/mpicoder.c: In function 'mpi_read_raw_from_sgl': /kisskb/src/lib/mpi/mpicoder.c:336:12: warning: 'buff' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/mm/mprotect.c: In function 'change_pte_range': /kisskb/src/mm/mprotect.c:42:20: warning: unused variable 'mm' [-Wunused-variable] /kisskb/src/net/core/gen_stats.c: In function '__gnet_stats_copy_basic': /kisskb/src/net/core/gen_stats.c:161:19: warning: 'seq' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/fs/proc/inode.c: In function 'proc_reg_open': /kisskb/src/include/linux/list.h:65:12: warning: 'pdeo' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/fs/proc/inode.c:337:21: note: 'pdeo' was declared here /kisskb/src/lib/rhashtable.c: In function 'rht_deferred_worker': /kisskb/src/lib/rhashtable.c:264:2: warning: 'next' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/lib/rhashtable.c:229:28: note: 'next' was declared here /kisskb/src/net/core/dev.c: In function 'validate_xmit_skb_list': /kisskb/src/net/core/dev.c:3405:15: warning: 'tail' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/net/core/filter.c: In function 'bpf_clear_redirect_map': /kisskb/src/net/core/filter.c:3479:4: warning: value computed is not used [-Wunused-value] /kisskb/src/fs/udf/unicode.c: In function 'udf_name_conv_char': /kisskb/src/fs/udf/unicode.c:132:8: warning: 'c' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/fs/posix_acl.c: In function 'get_acl': /kisskb/src/fs/posix_acl.c:147:3: warning: value computed is not used [-Wunused-value] /kisskb/src/fs/ocfs2/alloc.c: In function 'ocfs2_trim_mainbm': /kisskb/src/fs/ocfs2/alloc.c:7604:17: warning: 'first_bit' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/drivers/net/ethernet/i825xx/sun3_82586.c: In function 'sun3_82586_timeout': /kisskb/src/drivers/net/ethernet/i825xx/sun3_82586.c:993:89: warning: array subscript is above array bounds [-Warray-bounds] /kisskb/src/fs/ocfs2/file.c: In function 'ocfs2_file_write_iter': /kisskb/src/fs/ocfs2/file.c:2386:3: warning: value computed is not used [-Wunused-value] /kisskb/src/net/ipv4/ip_output.c: In function '__ip_append_data.isra.40': /kisskb/src/include/linux/skbuff.h:1397:6: warning: 'extra_uref' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/net/ipv4/ip_output.c:885:14: note: 'extra_uref' was declared here /kisskb/src/net/ipv6/ip6_output.c: In function '__ip6_append_data.isra.38': /kisskb/src/include/linux/skbuff.h:1397:6: warning: 'extra_uref' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/net/ipv6/ip6_output.c:1278:14: note: 'extra_uref' was declared here /kisskb/src/net/sctp/output.c: In function 'sctp_packet_config': /kisskb/src/include/net/sock.h:1949:19: warning: 'sk' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/net/sctp/output.c:94:15: note: 'sk' was declared here /kisskb/src/drivers/net/macsec.c: In function 'macsec_del_rxsa': /kisskb/src/drivers/net/macsec.c:1926:2: warning: 'assoc_num' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/drivers/net/macsec.c:1926:2: warning: 'rx_sc' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/drivers/net/macsec.c: In function 'macsec_del_txsa': /kisskb/src/drivers/net/macsec.c:2003:2: warning: 'assoc_num' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/drivers/net/macsec.c:2003:2: warning: 'tx_sc' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/drivers/net/macsec.c: In function 'macsec_upd_txsa': /kisskb/src/drivers/net/macsec.c:2068:5: warning: 'assoc_num' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/drivers/net/macsec.c:2068:24: warning: 'tx_sc' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/drivers/net/macsec.c:2069:21: warning: 'secy' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/drivers/net/tun.c: In function 'tun_get_user': /kisskb/src/drivers/net/tun.c:1842:30: warning: 'copylen' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/drivers/net/tun.c:1535:31: warning: 'linear' may be used uninitialized in this function [-Wuninitialized] /kisskb/src/drivers/net/tun.c:1755:46: note: 'linear' was declared here Completed OK # rm -rf /kisskb/build/linus_sun3_defconfig_m68k # Build took: 0:01:09.251417

© Michael Ellerman 2006-2018.