From karlkarpfen79 at gmail.com Thu Apr 9 12:50:01 2015 From: karlkarpfen79 at gmail.com (Karl Karpfen) Date: Thu, 9 Apr 2015 06:50:01 +0200 Subject: Dropbear for lwIP Message-ID: Hi, is it already possible to use Dropbear with the embedded TCP/IP stack lwIP? Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.ucc.gu.uwa.edu.au/pipermail/dropbear/attachments/20150409/4184084d/attachment.htm From loganaden at gmail.com Sun Apr 12 16:19:00 2015 From: loganaden at gmail.com (Loganaden Velvindron) Date: Sun, 12 Apr 2015 08:19:00 +0000 Subject: Harden dropbear memory allocator Message-ID: Hi All, OpenBSD introduced a new API known as reallocarray(): If malloc() must be used with multiplication, be sure to test for overflow: size_t num, size; ... /* Check for size_t overflow */ if (size && num > SIZE_MAX / size) errc(1, EOVERFLOW, "overflow"); if ((p = malloc(size * num)) == NULL) err(1, "malloc"); The above test is not sufficient in all cases. For example, multiplying ints requires a different set of checks: int num, size; ... /* Avoid invalid requests */ if (size < 0 || num < 0) errc(1, EOVERFLOW, "overflow"); /* Check for signed int overflow */ if (size && num > INT_MAX / size) errc(1, EOVERFLOW, "overflow"); if ((p = malloc(size * num)) == NULL) err(1, "malloc"); Assuming the implementation checks for integer overflow as OpenBSD does, it is much easier to use calloc() or reallocarray(). The dangers of multiplication integer overflows was what caused the one of the vulnerabilities in OpenSSH: https://www.owasp.org/index.php/Integer_overflow dropbear is one of the most popular ssh servers for home routers. I have conducted a analysis of dropbear's multiple m_malloc instances, and came up with the following diff: http://elandsys.com/~logan/dropbear_reallocarray_c.diff Feedback welcomed //Logan C-x-C-c From matt at ucc.asn.au Sun Apr 12 20:34:50 2015 From: matt at ucc.asn.au (Matt Johnston) Date: Sun, 12 Apr 2015 20:34:50 +0800 Subject: Harden dropbear memory allocator In-Reply-To: References: Message-ID: <20150412123450.GA3750@ucc.gu.uwa.edu.au> Hi Logan, Thanks for looking at that - certainly something to be wary of. I've actually removed netio.c's malloc altogether a couple of weeks ago though had neglected to push it until now https://secure.ucc.asn.au/hg/dropbear/rev/cc6116cc0b5c (my github mirror isn't updated, the machine that does that is offline, should be back next week). I think netio.c was OK because *ret_iov_count was limited to IOV_MAX which is 1024 or less on most platforms. Cheers, Matt On Sun, Apr 12, 2015 at 08:19:00AM +0000, Loganaden Velvindron wrote: > Hi All, > > OpenBSD introduced a new API known as reallocarray(): > > If malloc() must be used with multiplication, be sure to test for overflow: > > size_t num, size; > ... > > /* Check for size_t overflow */ > if (size && num > SIZE_MAX / size) > errc(1, EOVERFLOW, "overflow"); > > if ((p = malloc(size * num)) == NULL) > err(1, "malloc"); > > The above test is not sufficient in all cases. For example, > multiplying ints requires a different set of checks: > > int num, size; > ... > > /* Avoid invalid requests */ > if (size < 0 || num < 0) > errc(1, EOVERFLOW, "overflow"); > > /* Check for signed int overflow */ > if (size && num > INT_MAX / size) > errc(1, EOVERFLOW, "overflow"); > > if ((p = malloc(size * num)) == NULL) > err(1, "malloc"); > > Assuming the implementation checks for integer overflow as OpenBSD > does, it is much easier to use calloc() or reallocarray(). > > > The dangers of multiplication integer overflows was what caused the > one of the vulnerabilities in OpenSSH: > https://www.owasp.org/index.php/Integer_overflow > > dropbear is one of the most popular ssh servers for home routers. > > I have conducted a analysis of dropbear's multiple m_malloc instances, > and came up with the following diff: > > http://elandsys.com/~logan/dropbear_reallocarray_c.diff > > Feedback welcomed > //Logan > C-x-C-c From matt at ucc.asn.au Sun Apr 12 20:35:19 2015 From: matt at ucc.asn.au (Matt Johnston) Date: Sun, 12 Apr 2015 20:35:19 +0800 Subject: Dropbear for lwIP In-Reply-To: References: Message-ID: <20150412123519.GB3750@ucc.gu.uwa.edu.au> On Thu, Apr 09, 2015 at 06:50:01AM +0200, Karl Karpfen wrote: > Hi, > > is it already possible to use Dropbear with the embedded TCP/IP stack lwIP? Hi Karl, I haven't heard of anyone doing that. Cheers, Matt From loganaden at gmail.com Sun Apr 12 20:52:44 2015 From: loganaden at gmail.com (Loganaden Velvindron) Date: Sun, 12 Apr 2015 12:52:44 +0000 Subject: Harden dropbear memory allocator In-Reply-To: <20150412123450.GA3750@ucc.gu.uwa.edu.au> References: <20150412123450.GA3750@ucc.gu.uwa.edu.au> Message-ID: On Sun, Apr 12, 2015 at 12:34 PM, Matt Johnston wrote: > Hi Logan, > > Thanks for looking at that - certainly something to be wary > of. I've actually removed netio.c's malloc altogether a > couple of weeks ago though had neglected to push it until > now https://secure.ucc.asn.au/hg/dropbear/rev/cc6116cc0b5c > (my github mirror isn't updated, the machine that does that is > offline, should be back next week). You're welcome ! > > I think netio.c was OK because *ret_iov_count was limited to > IOV_MAX which is 1024 or less on most platforms. I was thinking about importing the m_reallocarray() wrapper into dropbear. I think that it would be a useful API to have inside dropbear for future use as well. It has the advantage of a low overhead compared to calloc() as it doesn't zero, and you have the integer overflow protection for free. For example, in scpmisc.c, There's also: args->list = xrealloc(args->list, nalloc * sizeof(char *)); In OpenBSD, we have replaced those calls in OpenBSD by reallocarray(), and there are around 800 instances: it ships in libressl, openssh, opensmtpd, and many other userland software, such as nsd and unbound, which have also imported a similar API into their memory wrappers. //Logan C-x-C-c From ed.sutter at alcatel-lucent.com Mon Apr 13 20:21:55 2015 From: ed.sutter at alcatel-lucent.com (Ed Sutter) Date: Mon, 13 Apr 2015 08:21:55 -0400 Subject: Dropbear for lwIP In-Reply-To: <20150412123519.GB3750@ucc.gu.uwa.edu.au> References: <20150412123519.GB3750@ucc.gu.uwa.edu.au> Message-ID: <552BB4E3.3010402@alcatel-lucent.com> On 4/12/2015 8:35 AM, Matt Johnston wrote: > On Thu, Apr 09, 2015 at 06:50:01AM +0200, Karl Karpfen wrote: >> Hi, >> >> is it already possible to use Dropbear with the embedded TCP/IP stack lwIP? > Hi Karl, > > I haven't heard of anyone doing that. > > Cheers, > Matt Karl, This is really a two-part problem... Aside from the stack api itself, I'm guessing since you're using lwIP you're likely running a smaller RTOS... There is code in Dropbear that assumes a linux/unix type of environment with fork and pseudottys, file system etc... A few years ago I had ported Dropbear into an environment that was running uC/OS-II with a TCP/IP stack that basically had the standard sockets API. That was tough enough. If your implementation of lwIP/TCP-IP is running in the state-machine mode, then that will be MUCH tougher. Ed -- Ed Sutter Alcatel-Lucent Technologies -- Bell Laboratories Phone: 908-582-2351 Email: ed.sutter at alcatel-lucent.com From ska-dietlibc at skarnet.org Wed Apr 22 22:39:43 2015 From: ska-dietlibc at skarnet.org (Laurent Bercot) Date: Wed, 22 Apr 2015 16:39:43 +0200 Subject: dbclient reports integrity errors with GitHub Message-ID: <5537B2AF.9010105@skarnet.org> Hello, dropbear-2015.67, compiled with a recent (last week) musl libc, statically linked. The problem also occurs with dropbear-2014.66. Since a few days ago (but I'm not sure when it started; in any case it's less than a month ago), every git pull and push over SSH from/to GitHub fails with the following message: dbclient: Connection to git at github.com:22 exited: Integrity error (bad packet size $N) $N is a nonsensical number, sometimes negative, but not always. Connections to GitHub via the OpenSSH client *work*. Connections to git repositories other than GitHub via dbclient *work*. Connections to GitHub via dbclient *do not work*, whether my client key is ECDSA or RSA. I don't use DSA. Any idea of what's going on or what I could do to learn more ? It's probably a GitHub problem, but I figured I had more chances of finding knowledgeable people here :) Thanks, -- Laurent From stevenhoneyman at gmail.com Wed Apr 22 23:16:36 2015 From: stevenhoneyman at gmail.com (Steven Honeyman) Date: Wed, 22 Apr 2015 16:16:36 +0100 Subject: dbclient reports integrity errors with GitHub In-Reply-To: <5537B2AF.9010105@skarnet.org> References: <5537B2AF.9010105@skarnet.org> Message-ID: On 22 April 2015 at 15:39, Laurent Bercot wrote: > > Hello, > dropbear-2015.67, compiled with a recent (last week) musl libc, > statically linked. > The problem also occurs with dropbear-2014.66. > > Since a few days ago (but I'm not sure when it started; in any > case it's less than a month ago), every git pull and push over SSH > from/to GitHub fails with the following message: > > dbclient: Connection to git at github.com:22 exited: Integrity error (bad > packet size $N) > > $N is a nonsensical number, sometimes negative, but not always. > > Connections to GitHub via the OpenSSH client *work*. > Connections to git repositories other than GitHub via dbclient *work*. > Connections to GitHub via dbclient *do not work*, whether my client > key is ECDSA or RSA. I don't use DSA. > > Any idea of what's going on or what I could do to learn more ? > It's probably a GitHub problem, but I figured I had more chances of > finding knowledgeable people here :) > > Thanks, > > -- > Laurent Hmm strange! I get the exact same behaviour with both musl and glibc, so it's likely a dropbear problem. Steven From matt at ucc.asn.au Thu Apr 23 07:46:05 2015 From: matt at ucc.asn.au (Matt Johnston) Date: Thu, 23 Apr 2015 07:46:05 +0800 Subject: dbclient reports integrity errors with GitHub In-Reply-To: References: <5537B2AF.9010105@skarnet.org> Message-ID: It sounds like something's going wrong with the cryptographic key setup. The packet size is the first thing decrypted so if they key is wrong the size will be wrong. I'll investigate what's going on, sounds like its easy to reproduce. Cheers, Matt On 22 April 2015 11:16:36 pm AWST, Steven Honeyman wrote: >On 22 April 2015 at 15:39, Laurent Bercot >wrote: >> >> Hello, >> dropbear-2015.67, compiled with a recent (last week) musl libc, >> statically linked. >> The problem also occurs with dropbear-2014.66. >> >> Since a few days ago (but I'm not sure when it started; in any >> case it's less than a month ago), every git pull and push over SSH >> from/to GitHub fails with the following message: >> >> dbclient: Connection to git at github.com:22 exited: Integrity error >(bad >> packet size $N) >> >> $N is a nonsensical number, sometimes negative, but not always. >> >> Connections to GitHub via the OpenSSH client *work*. >> Connections to git repositories other than GitHub via dbclient >*work*. >> Connections to GitHub via dbclient *do not work*, whether my client >> key is ECDSA or RSA. I don't use DSA. >> >> Any idea of what's going on or what I could do to learn more ? >> It's probably a GitHub problem, but I figured I had more chances of >> finding knowledgeable people here :) >> >> Thanks, >> >> -- >> Laurent > >Hmm strange! I get the exact same behaviour with both musl and glibc, >so it's likely a dropbear problem. > >Steven -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.ucc.gu.uwa.edu.au/pipermail/dropbear/attachments/20150423/eb1dabe0/attachment.htm From ska-dietlibc at skarnet.org Thu Apr 23 08:22:45 2015 From: ska-dietlibc at skarnet.org (Laurent Bercot) Date: Thu, 23 Apr 2015 02:22:45 +0200 Subject: dbclient reports integrity errors with GitHub In-Reply-To: <58C37F99-AC52-48CD-885A-B25B41F70BEB@ucc.asn.au> References: <5537B2AF.9010105@skarnet.org> <58C37F99-AC52-48CD-885A-B25B41F70BEB@ucc.asn.au> Message-ID: <55383B55.2070806@skarnet.org> On 23/04/2015 02:03, Matt Johnston wrote: > I think its because their version of libssh ignores one of the protocol flags, fixed in libssh 0.6.4. I'll send an email to github. > > https://git.libssh.org/projects/libssh.git/commit/?id=eb86fd8cdfd69f46c60bf0885a2125285e4a22b3 Thanks for your quick answer! I hope their reaction time will be as quick. (One can dream.) -- Laurent From gui at altermundi.net Thu May 7 06:58:04 2015 From: gui at altermundi.net (Gui Iribarren) Date: Wed, 06 May 2015 19:58:04 -0300 Subject: [REGRESSION] dbclient hangs forever when using agent forwarding since 2014.63 Message-ID: <554A9C7C.1070203@altermundi.net> I reported this a while back: https://www.mail-archive.com/dropbear%40ucc.asn.au/msg01287.html and finally got the time to do a git bisect over a clone of https://github.com/mkj/dropbear which yielded: 61cecbb3371839a824ff536338471d4b888aacf6 is the first bad commit commit 61cecbb3371839a824ff536338471d4b888aacf6 Author: Matt Johnston Date: Fri Jan 17 21:39:27 2014 +0800 DROPBEAR_CLI_AUTH_IMMEDIATE fixed, now enabled by default in a nutshell... using dropbear before that commit, i can happily use the ssh client (dbclient) in my openwrt router, to connect to a remote host, using a pubkey located in my laptop, through ssh agent forwarding i.e. [debian7]-->[openwrtA]-->[openwrtB] my pubkey is in debian7, dbclient 2013.62 is in openwrtA and i can ssh from debian7 into openwrtA, then do another ssh from openwrtA to openwrtB, without issues. The agent connection is forwarded correctly from debian7 to openwrtB by openwrtA's dbclient 2013.62 for auth purposes. since commit 61cecbb33, this no longer works as expected. dbclient hangs forever when trying to connect. i'm about to try "hotfix" it in my openwrt build, disabling CLI_IMMEDIATE_AUTH as suggested in: /* Send a real auth request immediately after sending a query for the available methods. * It saves a network round trip at login. * If problems are encountered it can be disabled here. */ #define CLI_IMMEDIATE_AUTH but, what would be the side-effects of disabling this? (or: what's the idea behind CLI_IMMEDIATE_AUTH?) maybe it was not "fixed" enough at the time it was committed / enabled by default? ;) (as it stands now, the regression is still present in latest release) thanks a lot for any pointers, with much respect for all your work, cheers! gui ================================================================= successful log, using dbclient compiled from git aee1309c9: gui at debian7:~ $ ssh openwrtA -vv -A [...] BusyBox v1.22.1 (2015-04-26 16:13:38 ART) built-in shell (ash) Enter 'help' for a list of built-in commands. root at openwrtA:~# ssh openwrtB debug1: client_input_channel_open: ctype auth-agent at openssh.com rchan 1 win 24576 max 32768 debug2: fd 9 setting O_NONBLOCK debug1: channel 3: new [authentication agent connection] debug1: confirm auth-agent at openssh.com debug2: channel 3: rcvd eof debug2: channel 3: output open -> drain debug2: channel 3: obuf empty debug2: channel 3: close_write debug2: channel 3: output drain -> closed debug1: channel 3: FORCE input drain debug2: channel 3: ibuf empty debug2: channel 3: send eof debug2: channel 3: input drain -> closed debug2: channel 3: rcvd close debug2: channel 3: send close debug2: channel 3: is dead debug2: channel 3: garbage collecting debug1: channel 3: free: authentication agent connection, nchannels 4 BusyBox v1.22.1 (2015-04-26 16:13:38 ART) built-in shell (ash) Enter 'help' for a list of built-in commands. root at openwrtB:~# ============================================================ unsuccessful log, using dbclient compiled from git 61cecbb33: gui at debian7:~ $ ssh openwrtA -vv -A [...] BusyBox v1.22.1 (2015-04-26 16:13:38 ART) built-in shell (ash) Enter 'help' for a list of built-in commands. root at openwrtA:~# ssh openwrtB debug1: client_input_channel_open: ctype auth-agent at openssh.com rchan 1 win 24576 max 32768 debug2: fd 9 setting O_NONBLOCK debug1: channel 3: new [authentication agent connection] debug1: confirm auth-agent at openssh.com [hangs forever] ============================================================ From gui at altermundi.net Thu May 7 07:08:30 2015 From: gui at altermundi.net (Gui Iribarren) Date: Wed, 06 May 2015 20:08:30 -0300 Subject: [REGRESSION] dbclient hangs forever when using agent forwarding since 2014.63 In-Reply-To: <554A9C7C.1070203@altermundi.net> References: <554A9C7C.1070203@altermundi.net> Message-ID: <554A9EEE.1020909@altermundi.net> On 06/05/15 19:58, Gui Iribarren wrote: > but, what would be the side-effects of disabling this? (or: what's the > idea behind CLI_IMMEDIATE_AUTH?) > maybe it was not "fixed" enough at the time it was committed / enabled > by default? ;) mmmh.. reading 2014.63 changelog: Fix DROPBEAR_CLI_IMMEDIATE_AUTH mode which saves a network round trip if the first public key succeeds. Still not enabled by default, needs more compatibility testing with other implementations i think it may be relevant to add that i have many pubkeys in my laptop, and it's quite probable that the *first* pubkey sent by the agent doesn't succeed in my case. cheers! From gui at altermundi.net Thu May 7 07:30:50 2015 From: gui at altermundi.net (Gui Iribarren) Date: Wed, 06 May 2015 20:30:50 -0300 Subject: [REGRESSION] dbclient hangs forever when using agent forwarding since 2014.63 In-Reply-To: <554A9EEE.1020909@altermundi.net> References: <554A9C7C.1070203@altermundi.net> <554A9EEE.1020909@altermundi.net> Message-ID: <554AA42A.3070401@altermundi.net> On 06/05/15 20:08, Gui Iribarren wrote: > On 06/05/15 19:58, Gui Iribarren wrote: >> but, what would be the side-effects of disabling this? (or: what's the >> idea behind CLI_IMMEDIATE_AUTH?) >> maybe it was not "fixed" enough at the time it was committed / enabled >> by default? ;) > > mmmh.. reading 2014.63 changelog: > > Fix DROPBEAR_CLI_IMMEDIATE_AUTH mode which saves a network round trip if > the first public key succeeds. Still not enabled by default, needs more > compatibility testing with other implementations humpf, well, indeed it is disabled by default since 2014.63 release; anyhow, it seems it doesn't make a difference whether DROPBEAR_CLI_IMMEDIATE_AUTH is enabled or not, the bug persists. in commit 61cecbb33 i can see code changes outside of #ifdef DROPBEAR_CLI_IMMEDIATE_AUTH but with my null C skills, this is as far as i dare to dig into this :) otherwise willing to help, gui > > i think it may be relevant to add that i have many pubkeys in my laptop, > and it's quite probable that the *first* pubkey sent by the agent > doesn't succeed in my case. > > cheers! > From guilhem at fripost.org Mon Jun 22 17:28:55 2015 From: guilhem at fripost.org (Guilhem Moulin) Date: Mon, 22 Jun 2015 11:28:55 +0200 Subject: [PATCH] Fix segfault with restricted authorized_key files without forced command. Message-ID: <1434965335-3119-1-git-send-email-guilhem@fripost.org> S $ sed -n '/ ssh-.*/{s///p; q}' ~/.ssh/authorized_keys no-port-forwarding S $ /usr/sbin/dropbear -r /tmp/dropbear.key -svEF -p 127.0.0.1:2222 [?] [6773] Jun 22 01:56:38 Port forwarding disabled. [?] [6773] Jun 22 01:56:38 Port forwarding disabled. [?] [6773] Jun 22 01:56:38 Pubkey auth succeeded for 'guilhem' with key ? TRACE (6773) 1434930998.973669: enter chansessionrequest TRACE (6773) 1434930998.973688: type is shell TRACE (6773) 1434930998.973712: enter sessioncommand Aiee, segfault! You should probably report this as a bug to the developer C $ ssh -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -p 2222 localhost Warning: Permanently added '[localhost]:2222' (RSA) to the list of known hosts. Connection to localhost closed by remote host. Connection to localhost closed. --- svr-authpubkeyoptions.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/svr-authpubkeyoptions.c b/svr-authpubkeyoptions.c index c296141..9bdf99d 100644 --- a/svr-authpubkeyoptions.c +++ b/svr-authpubkeyoptions.c @@ -91,7 +91,7 @@ int svr_pubkey_allows_pty() { /* Set chansession command to the one forced * by any 'command' public key option. */ void svr_pubkey_set_forced_command(struct ChanSess *chansess) { - if (ses.authstate.pubkey_options) { + if (ses.authstate.pubkey_options && ses.authstate.pubkey_options->forced_command) { if (chansess->cmd) { /* original_command takes ownership */ chansess->original_command = chansess->cmd; -- 2.1.4 From guilhem at fripost.org Mon Jun 22 20:44:30 2015 From: guilhem at fripost.org (Guilhem Moulin) Date: Mon, 22 Jun 2015 14:44:30 +0200 Subject: [PATCH] Fix typo in dropbear(8)'s manpage. Message-ID: <1434977070-8304-1-git-send-email-guilhem@fripost.org> --- dropbear.8 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/dropbear.8 b/dropbear.8 index 42f8ddb..3e05b08 100644 --- a/dropbear.8 +++ b/dropbear.8 @@ -100,8 +100,8 @@ Print the version .TP Authorized Keys -~/.ssh/authorized_keys can be set up to allow remote login with a RSA or DSS -key. Each line is of the form +~/.ssh/authorized_keys can be set up to allow remote login with a DSS, +RSA or ECDSA key. Each line is of the form .TP [restrictions] ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIgAsp... [comment] @@ -139,7 +139,7 @@ Host Key Files Host key files are read at startup from a standard location, by default /etc/dropbear/dropbear_dss_host_key, /etc/dropbear/dropbear_rsa_host_key, and -/etc/dropbear/dropbear-ecdsa_host_key +/etc/dropbear/dropbear_ecdsa_host_key or specified on the commandline with -r. These are of the form generated by dropbearkey. The -R option can be used to automatically generate keys in the default location - keys will be generated after startup when the first -- 2.1.4 From craig.mcqueen at innerrange.com Tue Jun 23 10:23:58 2015 From: craig.mcqueen at innerrange.com (Craig McQueen) Date: Tue, 23 Jun 2015 12:23:58 +1000 Subject: scp of sysfs file repeats data Message-ID: <5500469A22567C4BAF673A6E86AFA3A40227C1C14418@IR-CENTRAL.corp.innerrange.com> I tried doing scp of a sysfs attribute. I found that it duplicated the data, as well as adding extra NUL bytes. E.g.: # scp /sys/class/net/eth0/address localhost: # cat ~/address d0:39:72:53:ef:0e d0:39:72:53:ef:0e # hexdump -C ~/address 00000000 64 30 3a 33 39 3a 37 32 3a 35 33 3a 65 66 3a 30 |d0:39:72:53:ef:0| 00000010 65 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |e...............| 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 00000800 64 30 3a 33 39 3a 37 32 3a 35 33 3a 65 66 3a 30 |d0:39:72:53:ef:0| 00000810 65 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |e...............| 00000820 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 00001000 Does that seem normal? -- Craig McQueen From craig.mcqueen at innerrange.com Tue Jun 23 12:10:49 2015 From: craig.mcqueen at innerrange.com (Craig McQueen) Date: Tue, 23 Jun 2015 14:10:49 +1000 Subject: scp of sysfs file repeats data In-Reply-To: <5500469A22567C4BAF673A6E86AFA3A40227C1C14418@IR-CENTRAL.corp.innerrange.com> References: <5500469A22567C4BAF673A6E86AFA3A40227C1C14418@IR-CENTRAL.corp.innerrange.com> Message-ID: <5500469A22567C4BAF673A6E86AFA3A40227C1C14437@IR-CENTRAL.corp.innerrange.com> > Subject: scp of sysfs file repeats data > > I tried doing scp of a sysfs attribute. I found that it duplicated the data, as > well as adding extra NUL bytes. E.g.: > > # scp /sys/class/net/eth0/address localhost: > # cat ~/address > d0:39:72:53:ef:0e > d0:39:72:53:ef:0e > # hexdump -C ~/address > 00000000 64 30 3a 33 39 3a 37 32 3a 35 33 3a 65 66 3a 30 |d0:39:72:53:ef:0| > 00000010 65 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |e...............| > 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > * > 00000800 64 30 3a 33 39 3a 37 32 3a 35 33 3a 65 66 3a 30 |d0:39:72:53:ef:0| > 00000810 65 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |e...............| > 00000820 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > * > 00001000 > > Does that seem normal? By the way, this is on a BeagleBone Black-based (ARM Cortex) Linux system built with Yocto dizzy. From peter at korsgaard.com Tue Jun 23 15:11:35 2015 From: peter at korsgaard.com (Peter Korsgaard) Date: Tue, 23 Jun 2015 09:11:35 +0200 Subject: scp of sysfs file repeats data In-Reply-To: <5500469A22567C4BAF673A6E86AFA3A40227C1C14418@IR-CENTRAL.corp.innerrange.com> (Craig McQueen's message of "Tue, 23 Jun 2015 12:23:58 +1000") References: <5500469A22567C4BAF673A6E86AFA3A40227C1C14418@IR-CENTRAL.corp.innerrange.com> Message-ID: <87zj3q6grc.fsf@dell.be.48ers.dk> >>>>> "Craig" == Craig McQueen writes: > I tried doing scp of a sysfs attribute. I found that it duplicated the > data, as well as adding extra NUL bytes. E.g.: > # scp /sys/class/net/eth0/address localhost: > # cat ~/address > d0:39:72:53:ef:0e > d0:39:72:53:ef:0e > # hexdump -C ~/address > 00000000 64 30 3a 33 39 3a 37 32 3a 35 33 3a 65 66 3a 30 |d0:39:72:53:ef:0| > 00000010 65 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |e...............| > 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > * > 00000800 64 30 3a 33 39 3a 37 32 3a 35 33 3a 65 66 3a 30 |d0:39:72:53:ef:0| > 00000810 65 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |e...............| > 00000820 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > * > 00001000 > Does that seem normal? No, that looks like a bug. I see it as well here: strace output looks like this (for a sysfs property called control). open("control", O_RDONLY|O_LARGEFILE) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0 write(6, "C0644 4096 control\n", 19) = 19 read(3, "auto\n", 2048) = 5 read(3, "", 2043) = 0 write(6, "auto\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 2048) = 2048 read(3, "", 2048) = 0 write(6, "auto\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 2048) = 2048 The logic in scp.c always tries to copy st_size bytes, but the sysfs properties always claim to be 4096 while they in reality are shorter. I guess the loop in source() needs to be changed to end on error or eof instead. -- Bye, Peter Korsgaard From matt at ucc.asn.au Tue Jun 23 21:33:56 2015 From: matt at ucc.asn.au (Matt Johnston) Date: Tue, 23 Jun 2015 21:33:56 +0800 Subject: scp of sysfs file repeats data In-Reply-To: <87zj3q6grc.fsf@dell.be.48ers.dk> References: <5500469A22567C4BAF673A6E86AFA3A40227C1C14418@IR-CENTRAL.corp.innerrange.com> <87zj3q6grc.fsf@dell.be.48ers.dk> Message-ID: <20150623133356.GE15285@ucc.gu.uwa.edu.au> I see what you mean. I'll update scp to OpenSSH's latest - there are a few changes to be merged. Cheers, Matt On Tue, Jun 23, 2015 at 09:11:35AM +0200, Peter Korsgaard wrote: > >>>>> "Craig" == Craig McQueen writes: > > > I tried doing scp of a sysfs attribute. I found that it duplicated the > > data, as well as adding extra NUL bytes. E.g.: > > > # scp /sys/class/net/eth0/address localhost: > > # cat ~/address > > d0:39:72:53:ef:0e > > d0:39:72:53:ef:0e > > # hexdump -C ~/address > > 00000000 64 30 3a 33 39 3a 37 32 3a 35 33 3a 65 66 3a 30 |d0:39:72:53:ef:0| > > 00000010 65 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |e...............| > > 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > > * > > 00000800 64 30 3a 33 39 3a 37 32 3a 35 33 3a 65 66 3a 30 |d0:39:72:53:ef:0| > > 00000810 65 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |e...............| > > 00000820 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > > * > > 00001000 > > > Does that seem normal? > > No, that looks like a bug. I see it as well here: > > strace output looks like this (for a sysfs property called control). > > open("control", O_RDONLY|O_LARGEFILE) = 3 > fstat64(3, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0 > write(6, "C0644 4096 control\n", 19) = 19 > read(3, "auto\n", 2048) = 5 > read(3, "", 2043) = 0 > write(6, "auto\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 2048) = 2048 > read(3, "", 2048) = 0 > write(6, "auto\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 2048) = 2048 > > The logic in scp.c always tries to copy st_size bytes, but the sysfs > properties always claim to be 4096 while they in reality are shorter. > > I guess the loop in source() needs to be changed to end on error or eof > instead. > > -- > Bye, Peter Korsgaard From peter at korsgaard.com Tue Jun 23 23:41:20 2015 From: peter at korsgaard.com (Peter Korsgaard) Date: Tue, 23 Jun 2015 17:41:20 +0200 Subject: scp of sysfs file repeats data In-Reply-To: <20150623133356.GE15285@ucc.gu.uwa.edu.au> (Matt Johnston's message of "Tue, 23 Jun 2015 21:33:56 +0800") References: <5500469A22567C4BAF673A6E86AFA3A40227C1C14418@IR-CENTRAL.corp.innerrange.com> <87zj3q6grc.fsf@dell.be.48ers.dk> <20150623133356.GE15285@ucc.gu.uwa.edu.au> Message-ID: <87mvzq5t5r.fsf@dell.be.48ers.dk> >>>>> "Matt" == Matt Johnston writes: > I see what you mean. I'll update scp to OpenSSH's latest - > there are a few changes to be merged. FYI, openssh scp has a similar problem here: open("vendor", O_RDONLY|O_NONBLOCK) = 3 fstat(3, {st_mode=S_IFREG|0444, st_size=4096, ...}) = 0 fcntl(3, F_GETFL) = 0x8800 (flags O_RDONLY|O_NONBLOCK|O_LARGEFILE) fcntl(3, F_SETFL, O_RDONLY|O_LARGEFILE) = 0 write(6, "C0444 4096 vendor\n", 18) = 18 read(3, "0x8086\n", 4096) = 7 read(3, "", 4089) = 0 write(6, "0x8086\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 This is with openssh 6.7p1-6. -- Bye, Peter Korsgaard From peter at korsgaard.com Wed Jun 24 02:11:29 2015 From: peter at korsgaard.com (Peter Korsgaard) Date: Tue, 23 Jun 2015 20:11:29 +0200 Subject: scp of sysfs file repeats data In-Reply-To: <87mvzq5t5r.fsf@dell.be.48ers.dk> (Peter Korsgaard's message of "Tue, 23 Jun 2015 17:41:20 +0200") References: <5500469A22567C4BAF673A6E86AFA3A40227C1C14418@IR-CENTRAL.corp.innerrange.com> <87zj3q6grc.fsf@dell.be.48ers.dk> <20150623133356.GE15285@ucc.gu.uwa.edu.au> <87mvzq5t5r.fsf@dell.be.48ers.dk> Message-ID: <87ioae5m7i.fsf@dell.be.48ers.dk> >>>>> "Peter" == Peter Korsgaard writes: >>>>> "Matt" == Matt Johnston writes: >> I see what you mean. I'll update scp to OpenSSH's latest - >> there are a few changes to be merged. > FYI, openssh scp has a similar problem here: > open("vendor", O_RDONLY|O_NONBLOCK) = 3 > fstat(3, {st_mode=S_IFREG|0444, st_size=4096, ...}) = 0 > fcntl(3, F_GETFL) = 0x8800 (flags O_RDONLY|O_NONBLOCK|O_LARGEFILE) > fcntl(3, F_SETFL, O_RDONLY|O_LARGEFILE) = 0 > write(6, "C0444 4096 vendor\n", 18) = 18 > read(3, "0x8086\n", 4096) = 7 > read(3, "", 4089) = 0 > write(6, "0x8086\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096 > This is with openssh 6.7p1-6. But thinking further about it, I guess that's just an artifact of the protocol and there isn't any way around it. -- Bye, Peter Korsgaard From craig.mcqueen at innerrange.com Wed Jun 24 07:49:54 2015 From: craig.mcqueen at innerrange.com (Craig McQueen) Date: Wed, 24 Jun 2015 09:49:54 +1000 Subject: scp of sysfs file repeats data In-Reply-To: <87ioae5m7i.fsf@dell.be.48ers.dk> References: <5500469A22567C4BAF673A6E86AFA3A40227C1C14418@IR-CENTRAL.corp.innerrange.com> <87zj3q6grc.fsf@dell.be.48ers.dk> <20150623133356.GE15285@ucc.gu.uwa.edu.au> <87mvzq5t5r.fsf@dell.be.48ers.dk> <87ioae5m7i.fsf@dell.be.48ers.dk> Message-ID: <5500469A22567C4BAF673A6E86AFA3A40227C1C144BE@IR-CENTRAL.corp.innerrange.com> > From: Peter Korsgaard [mailto:jacmet at gmail.com] On Behalf Of Peter > > >>>>> "Peter" == Peter Korsgaard writes: > > >>>>> "Matt" == Matt Johnston writes: > >> I see what you mean. I'll update scp to OpenSSH's latest - >> there are a > few changes to be merged. > > > FYI, openssh scp has a similar problem here: > > > open("vendor", O_RDONLY|O_NONBLOCK) = 3 > > fstat(3, {st_mode=S_IFREG|0444, st_size=4096, ...}) = 0 > > fcntl(3, F_GETFL) = 0x8800 (flags > O_RDONLY|O_NONBLOCK|O_LARGEFILE) > > fcntl(3, F_SETFL, O_RDONLY|O_LARGEFILE) = 0 > > write(6, "C0444 4096 vendor\n", 18) = 18 > > read(3, "0x8086\n", 4096) = 7 > > read(3, "", 4089) = 0 > > write(6, > "0x8086\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = > 4096 > > > This is with openssh 6.7p1-6. > > But thinking further about it, I guess that's just an artifact of the protocol and > there isn't any way around it. I guess the question is, should the copied file's size be that reported in the directory listing, or what is actually successfully read from the file? If I use 'cp' to copy a sysfs file, then the copy contains the bytes actually read. E.g.: # ls -al /sys/class/net/eth0/address -r--r--r-- 1 root root 4096 Jun 23 23:43 /sys/class/net/eth0/address # cp /sys/class/net/eth0/address . # ls -al drwxr-xr-x 1 root root 4096 Jun 23 23:43 . drwxr-xr-x 1 root root 4096 Jun 23 06:51 .. -r--r--r-- 1 root root 18 Jun 23 23:43 address So with 'cp' I get a file length of 18 (the bytes successfully read), not 4096. As a user, I expect 'scp' to do the same thing. Does the protocol make that difficult somehow? -- Craig McQueen From Reimar.Doeffinger at gmx.de Wed Jun 24 10:23:19 2015 From: Reimar.Doeffinger at gmx.de (=?utf-8?Q?Reimar_D=C3=B6ffinger?=) Date: Wed, 24 Jun 2015 04:23:19 +0200 Subject: scp of sysfs file repeats data In-Reply-To: <5500469A22567C4BAF673A6E86AFA3A40227C1C144BE@IR-CENTRAL.corp.innerrange.com> References: <5500469A22567C4BAF673A6E86AFA3A40227C1C14418@IR-CENTRAL.corp.innerrange.com> <87zj3q6grc.fsf@dell.be.48ers.dk> <20150623133356.GE15285@ucc.gu.uwa.edu.au> <87mvzq5t5r.fsf@dell.be.48ers.dk> <87ioae5m7i.fsf@dell.be.48ers.dk> <5500469A22567C4BAF673A6E86AFA3A40227C1C144BE@IR-CENTRAL.corp.innerrange.com> Message-ID: <9DA49DF0-2AF6-43C8-AA91-D688457413CA@gmx.de> On 24.06.2015, at 01:49, Craig McQueen wrote: >> From: Peter Korsgaard [mailto:jacmet at gmail.com] On Behalf Of Peter >> >>>>>>> "Peter" == Peter Korsgaard writes: >> >>>>>>> "Matt" == Matt Johnston writes: >>>> I see what you mean. I'll update scp to OpenSSH's latest - >> there are a >> few changes to be merged. >> >>> FYI, openssh scp has a similar problem here: >> >>> open("vendor", O_RDONLY|O_NONBLOCK) = 3 >>> fstat(3, {st_mode=S_IFREG|0444, st_size=4096, ...}) = 0 >>> fcntl(3, F_GETFL) = 0x8800 (flags >> O_RDONLY|O_NONBLOCK|O_LARGEFILE) >>> fcntl(3, F_SETFL, O_RDONLY|O_LARGEFILE) = 0 >>> write(6, "C0444 4096 vendor\n", 18) = 18 >>> read(3, "0x8086\n", 4096) = 7 >>> read(3, "", 4089) = 0 >>> write(6, >> "0x8086\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = >> 4096 >> >>> This is with openssh 6.7p1-6. >> >> But thinking further about it, I guess that's just an artifact of the protocol and >> there isn't any way around it. > > I guess the question is, should the copied file's size be that reported in the directory listing, or what is actually successfully read from the file? > > If I use 'cp' to copy a sysfs file, then the copy contains the bytes actually read. E.g.: > > # ls -al /sys/class/net/eth0/address > -r--r--r-- 1 root root 4096 Jun 23 23:43 /sys/class/net/eth0/address > # cp /sys/class/net/eth0/address . > # ls -al > drwxr-xr-x 1 root root 4096 Jun 23 23:43 . > drwxr-xr-x 1 root root 4096 Jun 23 06:51 .. > -r--r--r-- 1 root root 18 Jun 23 23:43 address > > So with 'cp' I get a file length of 18 (the bytes successfully read), not 4096. > > As a user, I expect 'scp' to do the same thing. Does the protocol make that difficult somehow? Well, as I understand it the file size is sent first. If you cannot trust stat, you'd have read the whole file first and even buffer it (in case it changes - though I guess that race condition also exists with current stat usage). I see a few things that could (and probably should) be done: 1) if it works better, use lseek to end to get file size instead of stat 2) when read fails, zero the buffer (so at least there are only extra 0s, not 'random' data) 3) if the file size is less than some limit (64kB? - would be easier to make this the same as the normal buffer size, 4kB is a bit small anyway) do not rely on stat but just read the whole file to get the size without race conditions. Could be done by just always before anything else reading 64kB and if the read is short just skip anything else. 4) print a warning when read fails (if the protocol allows it) Sorry for the long text, especially as I have no intention to actually work on it. Hope someone is motivated to fix it :) Regards, Reimar From craig.mcqueen at innerrange.com Wed Jun 24 11:35:42 2015 From: craig.mcqueen at innerrange.com (Craig McQueen) Date: Wed, 24 Jun 2015 13:35:42 +1000 Subject: scp of sysfs file repeats data In-Reply-To: <9DA49DF0-2AF6-43C8-AA91-D688457413CA@gmx.de> References: <5500469A22567C4BAF673A6E86AFA3A40227C1C14418@IR-CENTRAL.corp.innerrange.com> <87zj3q6grc.fsf@dell.be.48ers.dk> <20150623133356.GE15285@ucc.gu.uwa.edu.au> <87mvzq5t5r.fsf@dell.be.48ers.dk> <87ioae5m7i.fsf@dell.be.48ers.dk> <5500469A22567C4BAF673A6E86AFA3A40227C1C144BE@IR-CENTRAL.corp.innerrange.com> <9DA49DF0-2AF6-43C8-AA91-D688457413CA@gmx.de> Message-ID: <5500469A22567C4BAF673A6E86AFA3A40227C1C14536@IR-CENTRAL.corp.innerrange.com> > From: Reimar D?ffinger [mailto:Reimar.Doeffinger at gmx.de] > > On 24.06.2015, at 01:49, Craig McQueen > wrote: > > >> From: Peter Korsgaard [mailto:jacmet at gmail.com] On Behalf Of Peter > >> > >>>>>>> "Peter" == Peter Korsgaard writes: > >> > >>>>>>> "Matt" == Matt Johnston writes: > >>>> I see what you mean. I'll update scp to OpenSSH's latest - >> > >>>> there are a > >> few changes to be merged. > >> > >>> FYI, openssh scp has a similar problem here: > >> > >>> open("vendor", O_RDONLY|O_NONBLOCK) = 3 > >>> fstat(3, {st_mode=S_IFREG|0444, st_size=4096, ...}) = 0 > >>> fcntl(3, F_GETFL) = 0x8800 (flags > >> O_RDONLY|O_NONBLOCK|O_LARGEFILE) > >>> fcntl(3, F_SETFL, O_RDONLY|O_LARGEFILE) = 0 > >>> write(6, "C0444 4096 vendor\n", 18) = 18 > >>> read(3, "0x8086\n", 4096) = 7 > >>> read(3, "", 4089) = 0 > >>> write(6, > >> "0x8086\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > >> 4096) = > >> 4096 > >> > >>> This is with openssh 6.7p1-6. > >> > >> But thinking further about it, I guess that's just an artifact of the > >> protocol and there isn't any way around it. > > > > I guess the question is, should the copied file's size be that reported in the > directory listing, or what is actually successfully read from the file? > > > > If I use 'cp' to copy a sysfs file, then the copy contains the bytes actually > read. E.g.: > > > > # ls -al /sys/class/net/eth0/address > > -r--r--r-- 1 root root 4096 Jun 23 23:43 /sys/class/net/eth0/address > > # cp /sys/class/net/eth0/address . > > # ls -al > > drwxr-xr-x 1 root root 4096 Jun 23 23:43 . > > drwxr-xr-x 1 root root 4096 Jun 23 06:51 .. > > -r--r--r-- 1 root root 18 Jun 23 23:43 address > > > > So with 'cp' I get a file length of 18 (the bytes successfully read), not 4096. > > > > As a user, I expect 'scp' to do the same thing. Does the protocol make that > difficult somehow? > > Well, as I understand it the file size is sent first. > > If you cannot trust stat, you'd have read the whole file first and even buffer it > (in case it changes - though I guess that race condition also exists with current > stat usage). So in that case, 'scp' wouldn't work well with: * Files that are appended or truncated while they're being 'scp'ed. * sysfs files. * procfs files. * Named pipes. * char devices. It does sound as though the protocol was designed only with static regular files in mind. That's a pity. > I see a few things that could (and probably should) be done: > 1) if it works better, use lseek to end to get file size instead of stat Not sure if that would work better. > 2) when read fails, zero the buffer (so at least there are only extra 0s, not > 'random' data) That seems important, otherwise you could have a security issue vaguely like POODLE. > 3) if the file size is less than some limit (64kB? - would be easier to make this > the same as the normal buffer size, 4kB is a bit small anyway) do not rely on > stat but just read the whole file to get the size without race conditions. Could > be done by just always before anything else reading 64kB and if the read is > short just skip anything else. That would fix some use-cases, but perhaps make other use-case failures more obscure and surprising. > 4) print a warning when read fails (if the protocol allows it) Warnings are good as long as there is a human running it who can read it; not so good if it's a component of a larger software system. > Sorry for the long text, especially as I have no intention to actually work on it. > Hope someone is motivated to fix it :) It would be great if the situation could be improved. It is certainly best for users to be able to trust essential system tools like scp. Having file size sent in the protocol up-front does sound like an awkward protocol design, which cramps the solution space, unfortunately. -- Craig McQueen From Reimar.Doeffinger at gmx.de Wed Jun 24 12:46:45 2015 From: Reimar.Doeffinger at gmx.de (=?utf-8?Q?Reimar_D=C3=B6ffinger?=) Date: Wed, 24 Jun 2015 06:46:45 +0200 Subject: scp of sysfs file repeats data In-Reply-To: <5500469A22567C4BAF673A6E86AFA3A40227C1C144BE@IR-CENTRAL.corp.innerrange.com> References: <5500469A22567C4BAF673A6E86AFA3A40227C1C14418@IR-CENTRAL.corp.innerrange.com> <87zj3q6grc.fsf@dell.be.48ers.dk> <20150623133356.GE15285@ucc.gu.uwa.edu.au> <87mvzq5t5r.fsf@dell.be.48ers.dk> <87ioae5m7i.fsf@dell.be.48ers.dk> <5500469A22567C4BAF673A6E86AFA3A40227C1C144BE@IR-CENTRAL.corp.innerrange.com> Message-ID: On 24.06.2015, at 01:49, Craig McQueen wrote: >> From: Peter Korsgaard [mailto:jacmet at gmail.com] On Behalf Of Peter >> >>>>>>> "Peter" == Peter Korsgaard writes: >> >>>>>>> "Matt" == Matt Johnston writes: >>>> I see what you mean. I'll update scp to OpenSSH's latest - >> there are a >> few changes to be merged. >> >>> FYI, openssh scp has a similar problem here: >> >>> open("vendor", O_RDONLY|O_NONBLOCK) = 3 >>> fstat(3, {st_mode=S_IFREG|0444, st_size=4096, ...}) = 0 >>> fcntl(3, F_GETFL) = 0x8800 (flags >> O_RDONLY|O_NONBLOCK|O_LARGEFILE) >>> fcntl(3, F_SETFL, O_RDONLY|O_LARGEFILE) = 0 >>> write(6, "C0444 4096 vendor\n", 18) = 18 >>> read(3, "0x8086\n", 4096) = 7 >>> read(3, "", 4089) = 0 >>> write(6, >> "0x8086\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = >> 4096 >> >>> This is with openssh 6.7p1-6. Looking at the source code, the protocol should have sent an EIO error, so the user/programs checking the exit code should be able to see something went wrong... Is that somehow not working? From craig.mcqueen at innerrange.com Wed Jun 24 12:52:19 2015 From: craig.mcqueen at innerrange.com (Craig McQueen) Date: Wed, 24 Jun 2015 14:52:19 +1000 Subject: scp of sysfs file repeats data In-Reply-To: References: <5500469A22567C4BAF673A6E86AFA3A40227C1C14418@IR-CENTRAL.corp.innerrange.com> <87zj3q6grc.fsf@dell.be.48ers.dk> <20150623133356.GE15285@ucc.gu.uwa.edu.au> <87mvzq5t5r.fsf@dell.be.48ers.dk> <87ioae5m7i.fsf@dell.be.48ers.dk> <5500469A22567C4BAF673A6E86AFA3A40227C1C144BE@IR-CENTRAL.corp.innerrange.com> Message-ID: <5500469A22567C4BAF673A6E86AFA3A40227C1C1455E@IR-CENTRAL.corp.innerrange.com> > From: Reimar D?ffinger [mailto:Reimar.Doeffinger at gmx.de] > > On 24.06.2015, at 01:49, Craig McQueen > wrote: > >> From: Peter Korsgaard [mailto:jacmet at gmail.com] On Behalf Of Peter > >> > >>>>>>> "Peter" == Peter Korsgaard writes: > >> > >>>>>>> "Matt" == Matt Johnston writes: > >>>> I see what you mean. I'll update scp to OpenSSH's latest - >> there are > a > >> few changes to be merged. > >> > >>> FYI, openssh scp has a similar problem here: > >> > >>> open("vendor", O_RDONLY|O_NONBLOCK) = 3 > >>> fstat(3, {st_mode=S_IFREG|0444, st_size=4096, ...}) = 0 > >>> fcntl(3, F_GETFL) = 0x8800 (flags > >> O_RDONLY|O_NONBLOCK|O_LARGEFILE) > >>> fcntl(3, F_SETFL, O_RDONLY|O_LARGEFILE) = 0 > >>> write(6, "C0444 4096 vendor\n", 18) = 18 > >>> read(3, "0x8086\n", 4096) = 7 > >>> read(3, "", 4089) = 0 > >>> write(6, > >> "0x8086\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) > = > >> 4096 > >> > >>> This is with openssh 6.7p1-6. > > Looking at the source code, the protocol should have sent an EIO error, so > the user/programs checking the exit code should be able to see something > went wrong... > Is that somehow not working? Yes, with openssh scp, I get a "Broken pipe" error. That's good under the circumstances. With dropbear scp, I don't get any error. -- Craig McQueen From Reimar.Doeffinger at gmx.de Wed Jun 24 13:02:05 2015 From: Reimar.Doeffinger at gmx.de (Reimar =?iso-8859-1?Q?D=F6ffinger?=) Date: Wed, 24 Jun 2015 07:02:05 +0200 Subject: scp of sysfs file repeats data In-Reply-To: <9DA49DF0-2AF6-43C8-AA91-D688457413CA@gmx.de> References: <5500469A22567C4BAF673A6E86AFA3A40227C1C14418@IR-CENTRAL.corp.innerrange.com> <87zj3q6grc.fsf@dell.be.48ers.dk> <20150623133356.GE15285@ucc.gu.uwa.edu.au> <87mvzq5t5r.fsf@dell.be.48ers.dk> <87ioae5m7i.fsf@dell.be.48ers.dk> <5500469A22567C4BAF673A6E86AFA3A40227C1C144BE@IR-CENTRAL.corp.innerrange.com> <9DA49DF0-2AF6-43C8-AA91-D688457413CA@gmx.de> Message-ID: <20150624050205.GA3870@reimardoeffinger.de> On Wed, Jun 24, 2015 at 04:23:19AM +0200, Reimar D?ffinger wrote: > 2) when read fails, zero the buffer (so at least there are only extra 0s, not 'random' data) That is done in OpenSSH CVS version at least. This seems to have changed in June 2014. From bernd at sysprog.at Fri Jun 26 15:47:21 2015 From: bernd at sysprog.at (Bernd Petrovitsch) Date: Fri, 26 Jun 2015 09:47:21 +0200 Subject: scp of sysfs file repeats data In-Reply-To: <5500469A22567C4BAF673A6E86AFA3A40227C1C1455E@IR-CENTRAL.corp.innerrange.com> References: <5500469A22567C4BAF673A6E86AFA3A40227C1C14418@IR-CENTRAL.corp.innerrange.com> <87zj3q6grc.fsf@dell.be.48ers.dk> <20150623133356.GE15285@ucc.gu.uwa.edu.au> <87mvzq5t5r.fsf@dell.be.48ers.dk> <87ioae5m7i.fsf@dell.be.48ers.dk> <5500469A22567C4BAF673A6E86AFA3A40227C1C144BE@IR-CENTRAL.corp.innerrange.com> <5500469A22567C4BAF673A6E86AFA3A40227C1C1455E@IR-CENTRAL.corp.innerrange.com> Message-ID: <1435304842.3337.79.camel@thorin> On Mit, 2015-06-24 at 14:52 +1000, Craig McQueen wrote: > > From: Reimar D?ffinger [mailto:Reimar.Doeffinger at gmx.de] [...] > > Looking at the source code, the protocol should have sent an EIO error, so > > the user/programs checking the exit code should be able to see something > > went wrong... > > Is that somehow not working? > > Yes, with openssh scp, I get a "Broken pipe" error. That's good under the circumstances. > > With dropbear scp, I don't get any error. Well, the sending side could check the sent size and the real size and write an error message - doesn't solve anything but at least one has the chance to see it (and use 'tar -zc | ssh tar -zx' instead or other already working means). Kind regards, Bernd -- "I dislike type abstraction if it has no real reason. And saving on typing is not a good reason - if your typing speed is the main issue when you're coding, you're doing something seriously wrong." - Linus Torvalds From guilhem at fripost.org Mon Jun 29 00:02:01 2015 From: guilhem at fripost.org (Guilhem Moulin) Date: Sun, 28 Jun 2015 18:02:01 +0200 Subject: Detached tarball signatures vs. clearsigned checksum files Message-ID: <20150628160200.GA28437@localhost> Hi Matt, I'm currently helping out packaging dropbear for Debian [0]. As mentioned on your webpage the drobpear package is currently rather outdated (even sid is lagging behind with 2014.65-1), and in order to reduce the delays between upstream and package releases I'd like to make the import of upstream tarballs easier. Along with the most recent tarballs, one finds a clearsigned SHA256SUM.asc file in https://matt.ucc.asn.au/dropbear/ . Since sha256sum(1) chokes on the OpenPGP header, in order to verify the integrity of the package one needs to 1/ run `gpg --verify`, 2/ remove the OpenPGP header & footer, and 3/ run `sha256sum -c`. I wonder if you could provide a detached signature of the tarball instead of clearsigning the checksum file. While Debian's uscan(1) is currently not able to deal with checksum files, it can import detached signatures along with tarballs and check the signature validity. (Furthermore it doesn't rely on the WoT since the signer's key is available in the repository under ?debian/upstream/signing-key.asc?.) This would make importing further releases much easier :-) In a nutshell this is what I have in mind: ./dropbear-2015.67.tar.bz2 ./dropbear-2015.67.tar.bz2.sig (or .asc for armored files) ./SHA256SUM (optional) Also risking nitpicking, you could also modify your gpg(1) digest preferences to something stronger than SHA1 [1] :-P For instance: echo 'personal-digest-preferences SHA512' >> ~/.gnupg/gpg.conf Thanks! Cheers, -- Guilhem. [0] https://lists.debian.org/debian-devel/2015/06/msg00285.html [1] https://www.debian-administration.org/users/dkg/weblog/48 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: Digital signature Url : http://lists.ucc.gu.uwa.edu.au/pipermail/dropbear/attachments/20150628/ca6ba1b0/attachment.sig From matt at ucc.asn.au Mon Jun 29 21:27:23 2015 From: matt at ucc.asn.au (Matt Johnston) Date: Mon, 29 Jun 2015 21:27:23 +0800 Subject: Detached tarball signatures vs. clearsigned checksum files In-Reply-To: <20150628160200.GA28437@localhost> References: <20150628160200.GA28437@localhost> Message-ID: <20150629132723.GH15285@ucc.gu.uwa.edu.au> On Sun, Jun 28, 2015 at 06:02:01PM +0200, Guilhem Moulin wrote: > I'm currently helping out packaging dropbear for Debian [0]. As > mentioned on your webpage the drobpear package is currently rather > outdated (even sid is lagging behind with 2014.65-1), and in order to > reduce the delays between upstream and package releases I'd like to make > the import of upstream tarballs easier. > This would make importing further releases much easier :-) In a > nutshell this is what I have in mind: > > ./dropbear-2015.67.tar.bz2 > ./dropbear-2015.67.tar.bz2.sig (or .asc for armored files) > ./SHA256SUM (optional) > Also risking nitpicking, you could also modify your gpg(1) digest > preferences to something stronger than SHA1 [1] :-P For instance: Hi Guilhem, New Debian packages would be great. I've signed releases/dropbear-2015.67.tar.bz2.sig for the latest one so far, I'll keep more for future releases. Making a new pgp key has been on my todo list so there is now a Dropbear Release Key. (The old key is DSA so seemed to only make SHA1 signatures) https://matt.ucc.asn.au/dropbear/releases/dropbear-key-2015.asc pub 4096R/F29C6773 2015-06-29 Key fingerprint = F734 7EF2 EE2E 07A2 6762 8CA9 4493 1494 F29C 6773 uid Dropbear SSH Release Signing It's signed by the old key and my new personal key pub 4096R/C20BBAAC 2015-06-29 Key fingerprint = 1F1A F0BB EC7C F375 9FFA 1191 F498 3012 C20B BAAC uid Matt Johnston sub 4096R/D5581050 2015-06-29 Cheers, Matt From guilhem at fripost.org Mon Jun 29 21:51:54 2015 From: guilhem at fripost.org (Guilhem Moulin) Date: Mon, 29 Jun 2015 15:51:54 +0200 Subject: Detached tarball signatures vs. clearsigned checksum files In-Reply-To: <20150629132723.GH15285@ucc.gu.uwa.edu.au> References: <20150628160200.GA28437@localhost> <20150629132723.GH15285@ucc.gu.uwa.edu.au> Message-ID: <20150629135154.GA10344@localhost> Hi, On Mon, 29 Jun 2015 at 21:27:23 +0800, Matt Johnston wrote: > New Debian packages would be great. I've signed > releases/dropbear-2015.67.tar.bz2.sig for the latest > one so far, I'll keep more for future releases. > [?] > Making a new pgp key has been on my todo list so there is now > a Dropbear Release Key. (The old key is DSA so seemed to > only make SHA1 signatures) That's great, thanks! While I'm at it, please also consider excluding mercurial dotfiles from the tarballs: ************************************************************************ diff --git a/release.sh b/release.sh index f377d0e..f2c6cad 100755 --- a/release.sh +++ b/release.sh @@ -27,7 +27,7 @@ if test -e $ARCHIVE; then exit 1 fi -hg archive "$RELDIR" || exit 2 +hg archive "$RELDIR" -X ".hg*" || exit 2 (cd "$RELDIR" && autoconf && autoheader) || exit 2 ************************************************************************ (Not sure if you left the ?./debian? directory on purpose, but if not you might want to exclude it as well.) Cheers, -- Guilhem. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: Digital signature Url : http://lists.ucc.gu.uwa.edu.au/pipermail/dropbear/attachments/20150629/d7b1136f/attachment-0001.sig From matt at ucc.asn.au Mon Jun 29 22:06:20 2015 From: matt at ucc.asn.au (Matt Johnston) Date: Mon, 29 Jun 2015 22:06:20 +0800 Subject: Detached tarball signatures vs. clearsigned checksum files In-Reply-To: <20150629135154.GA10344@localhost> References: <20150628160200.GA28437@localhost> <20150629132723.GH15285@ucc.gu.uwa.edu.au> <20150629135154.GA10344@localhost> Message-ID: <20150629140620.GJ15285@ucc.gu.uwa.edu.au> On Mon, Jun 29, 2015 at 03:51:54PM +0200, Guilhem Moulin wrote: > That's great, thanks! While I'm at it, please also consider excluding > mercurial dotfiles from the tarballs: Do they cause a problem? At least hg_archival.txt is kind of useful to see which hg revision made the tarball. > (Not sure if you left the ?./debian? directory on purpose, but if not > you might want to exclude it as well.) I build debs myself for my own use so I'll leave it there. Cheers, Matt From andrea.bola at yahoo.it Mon Jun 29 22:13:44 2015 From: andrea.bola at yahoo.it (Andrea Bolandrina) Date: Mon, 29 Jun 2015 15:13:44 +0100 Subject: Detached tarball signatures vs. clearsigned checksum files In-Reply-To: <20150629140620.GJ15285@ucc.gu.uwa.edu.au> References: <20150628160200.GA28437@localhost> <20150629132723.GH15285@ucc.gu.uwa.edu.au> <20150629135154.GA10344@localhost> <20150629140620.GJ15285@ucc.gu.uwa.edu.au> Message-ID: Hello, how do I remove myself from this mailing list? There is no link at the bottom (or anywhere else)... On Mon, Jun 29, 2015 at 3:06 PM, Matt Johnston wrote: > On Mon, Jun 29, 2015 at 03:51:54PM +0200, Guilhem Moulin wrote: > > That's great, thanks! While I'm at it, please also consider excluding > > mercurial dotfiles from the tarballs: > > Do they cause a problem? At least hg_archival.txt is kind of > useful to see which hg revision made the tarball. > > > (Not sure if you left the ?./debian? directory on purpose, but if not > > you might want to exclude it as well.) > > I build debs myself for my own use so I'll leave it there. > > Cheers, > Matt > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.ucc.gu.uwa.edu.au/pipermail/dropbear/attachments/20150629/1e6cc0a9/attachment.htm From guilhem at fripost.org Mon Jun 29 22:21:33 2015 From: guilhem at fripost.org (Guilhem Moulin) Date: Mon, 29 Jun 2015 16:21:33 +0200 Subject: Mercurial dotfiles (Was: Detached tarball signatures vs. clearsigned checksum files) In-Reply-To: <20150629140620.GJ15285@ucc.gu.uwa.edu.au> References: <20150628160200.GA28437@localhost> <20150629132723.GH15285@ucc.gu.uwa.edu.au> <20150629135154.GA10344@localhost> <20150629140620.GJ15285@ucc.gu.uwa.edu.au> Message-ID: <20150629142133.GA20620@localhost> On Mon, 29 Jun 2015 at 22:06:20 +0800, Matt Johnston wrote: > On Mon, Jun 29, 2015 at 03:51:54PM +0200, Guilhem Moulin wrote: >> That's great, thanks! While I'm at it, please also consider excluding >> mercurial dotfiles from the tarballs: > > Do they cause a problem? At least hg_archival.txt is kind of > useful to see which hg revision made the tarball. Nah I guess they are harmless; now that you say it hg_archival.txt can be useful indeed, and in fact lintian(1) only complains about .hgtags: https://lintian.debian.org/tags/source-contains-hg-tags-file.html -- Guilhem. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: Digital signature Url : http://lists.ucc.gu.uwa.edu.au/pipermail/dropbear/attachments/20150629/725cdd2c/attachment.sig From guilhem at fripost.org Mon Jun 29 22:23:10 2015 From: guilhem at fripost.org (Guilhem Moulin) Date: Mon, 29 Jun 2015 16:23:10 +0200 Subject: Detached tarball signatures vs. clearsigned checksum files In-Reply-To: References: <20150628160200.GA28437@localhost> <20150629132723.GH15285@ucc.gu.uwa.edu.au> <20150629135154.GA10344@localhost> <20150629140620.GJ15285@ucc.gu.uwa.edu.au> Message-ID: <20150629142310.GB20620@localhost> On Mon, 29 Jun 2015 at 15:13:44 +0100, Andrea Bolandrina wrote: > how do I remove myself from this mailing list? > > There is no link at the bottom (or anywhere else)... Yes, not in the body but in the headers: List-Unsubscribe: , -- Guilhem. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: Digital signature Url : http://lists.ucc.gu.uwa.edu.au/pipermail/dropbear/attachments/20150629/9925aa3f/attachment.sig