Austin Group Defect Tracker

Aardvark Mark IV

Main | My View | View Issues | Change Log | Docs
  

Viewing Issue Simple Details [ Jump to Notes ] [ Issue History ] [ Print ]
ID Category Severity Type Date Submitted Last Update
0001079 [1003.1(2008)/Issue 7] Shell and Utilities Editorial Enhancement Request 2016-10-05 23:40 2018-04-12 15:12
Reporter tobiasm View Status public  
Assigned To ajosey
Priority normal Resolution Rejected  
Status Closed  
Name Tobias Martens
Organization
User Reference
Section bc
Page Number -
Line Number -
Interp Status ---
Final Accepted Text
Summary 0001079: focus on bc being an arithmetic language
Description Due to the specified precedence of operators in bc, where unary - is higher than ^ (exponentiation operator), the result of

-(1+1)^2

will be 4, unlike any other calculator or arithmetic language I have encountered. I see the advantage this may have, for example (since bc often takes external input) when piping variable to bc in shell:

var=-2
echo "${var}^2" | bc

will result in 4, which may seem intuitive and mathematical to someone who is not aware of how shell processes the string. Personally, while I encourage allowing a intuitive approach, this shall not influence common arithmetic as given by the example above.
Moreover, the apparent advantage is destroyed by a simple subtraction:

echo "3-${var}" | bc

produces a syntax error (in GNU bc 1.06.95), likely due to the specification of ++ and -- (unary increment and decrement). Note how the same command works for the corresponding addition.
Again, other arithmetic languages will compute the expression 3--2 correctly to 5, only bc, sticking with programming languages' syntax and behaviour, will not be able to do so.

Let me conclude with how bc is called in the specification:
"bc - arbitrary-precision arithmetic language"
Desired Action First I want to invite you to discuss a shift of bc to a more arithmetic language (in the common sense) with me, occurring problems and possibilities.

In my opinion, the following actions would move bc to a arithmetic language as known from calculators and other arithmetic languages all over the world:

-switch precedence of unary - and ^.

-remove unary increment and decrement.
This may sound drastic, but those operators are shortcut programming syntax, not arithemtic. There is no need for these shortcuts in bc because they aren't used nearly as often as in programming languages. They are redundant and only add unnecessary complexity and confusion when simple subtractions fail.
Tags No tags attached.
Attached Files

- Relationships

-  Notes
(0003395)
Vincent Lefevre (reporter)
2016-10-06 07:41
edited on: 2016-10-06 08:02

 Concerning the high precedence of the unary -, I don't like it either, but... Note that this precedence is also higher than the one of the *, / and % operators, which also doesn't match the usual precedence in purely mathematical expressions. However, I don't think that for the *, / and % operators, bc is currently affected, as these operations (taking truncation into account) seem symmetric. Note that there are similar issues in C with the unary - and the *, / and % operators, except that C is really affected due to the partial lack of symmetry (see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=36588 [^] where I proposed warnings suggesting parentheses). And if one wants consistency with C, the precedence should not change.

Concerning the parsing issue with increment and decrement, I disagree. You should put spaces around the binary - for more security. Note that gp (from the GP/PARI calculator) also complains:

? 3--2
  *** unused characters: 3--2
  *** ^---

Ditto with calc (arbitrary precision calculator - http://www.isthe.com/chongo/tech/comp/calc/ [^] ):

; 3--2
Bad ++ usage

And with Maple:

> 3--2;
syntax error, `-` unexpected:

And, of course, in C with: int f(void) { return 3--2; }
(0003396)
Vincent Lefevre (reporter)
2016-10-06 08:09

 Mathematica also complains about the use of --:

In[1]:= 3-- 2

During evaluation of In[1]:= Decrement::rvalue: 3 is not a variable with a value, so its value cannot be changed. >>

Out[1]= 2 3--

(the space after -- in the input is automatically added by the interactive UI, but the problem would be the same in a non-interactive way or with copy-paste).

Even when writing math expressions on a paper, I would never write 3--2, or even with spaces: 3 - -2. I would always put parentheses: 3 - (-2). As a language, it entirely depends on whether one would be allowed to add new operators like "--" (for whatever meaning).
(0003397)
stephane (reporter)
2016-10-06 10:19
edited on: 2016-10-06 10:39

 bc's behaviour is consistent with that of zsh/bash/ksh arithmetic operators or TCL's expr:

$ bash -c 'echo $((-(1+1)**2))'
4
$ ksh -c 'echo $((-(1+1)**2))'
4
$ zsh -c 'echo $((-(1+1)**2))'
4
$ expect -c 'puts [expr -(1+1) ** 2]'
4


I believe among shells, ** was first added in zsh (circa 1994). Initially it had the same precedence as */% (though was right-associative) and that was changed shortly after to have a higher precedence.

Note that bc was initially a wrapper around dc (and still is on some systems), that uses reverse polish notation, where there's no negation unary operator, but negative constants are entered as _4. That pseudo-negation-operator possibly contributed to the unary negation operator being seen as having the highest precedence among Unix users.

$ echo '_2 2 ^ p' | dc
4


(The unary negation would be 0r- in dc.)

$ echo '2 0r- 2 ^ p' | dc
4


"expr" is another utility that doesn't have a "-" unary operator (but you can enter negative constants as "-2"). It doesn't have an exponentiation operator but that's another Unix utility that re-enforces the fact that unary - has highest precedence.

I personally find awk/perl/python's behaviour of -2 ^ 2 returning -4 unintuitive. That makes it the only binary operator that has precedence over unary operators. That's even worse for ** (perl/python) that is shaped similarly to * (which has a much lower precedence than unary -), that is where it's really a binary operator, not a kind of affix.

Shell arithmetics have the same issue with "-" and "+" conflicting with "--" and "++" (whose support the spec explicitly allows but not mandate). That was discussed recently on the mailing list about $((-x)) not being the same as $((-$x)) (and the former being preferred as a result).

In anycase, IMO, a standard can't break backward compatibility in such a dramatic way.
(0003398)
tobiasm (reporter)
2016-10-06 23:05

 I don't see how switching only the precedence of unary - and ^ (no other precedence is affected) would break consistency with C, since C requires you to unambiguously set precedence yourself with pow().
I'm very grateful for your examples, regarding -(1+1)**2 one can also take a look at gnuplot, which will return -4. However, it isn't my intention to throw examples back and forth, pretty sure there are enough for both variants. The question to ask is whether we want bc to stick to a feature useful in and coming from reverse Polish notation, despite using Polish notation itself. I'll pursue this later, but want to address the other point first.


Regarding the distinction between unary decrement and subtraction of negative:
I totally agree that it's the best to put parentheses generally, but we also need to look what should happen if someone does not. The important difference between C (and languages using C-like syntax, e.g. the quoted pg and calc) on the one hand and bc on the other: The former mostly work with code, calculations are self contained. Something like 3--2 will never happen, because -2 is never separated in the first way. bc however often works with user input strings, where -- will occur! (See the example above: echo "3-$var" | bc). For single-line (and sometimes sloppy) user input imo, unlike code, a strict or very careful syntax with parentheses and spaces around each expression shouldn't be demanded.
Besides, we don't need to discuss if this general syntax should be allowed in bc because it already is - the specs mandate it. Just try

3+-2

bc resolve it correctly to '3 + (-2)'. The only problem now is the -- possible twofold interpretation, where bc will return error.
Note how bash at least has no problems calculating (so consistency is broken there):

bash -c 'echo $((3--2))'
5

The question here (kinda also applies to the previous point): Focus on some other programming languages' C-like syntax or common arithmetic?


Its pretty obvious that there won't be a statement what is right or wrong, this is about the orientation of bc. Looking at the purpose, I feel that unlike C or Mathematica bc is much more often confronted with user input in form of a single string.
Independent of the particular problems, I think that a standard either sets common understanding of how things should work, or should adhere to it if already settled. The latter applies, at least for user input specialized languages. Some examples, also to show what type of calculator I mean:
My old Casio (TI likely the same).
Google Search.
WolframAlpha (based on Mathematica and from the same company, but focused on single string user input).
Desktop Calculators.
Each will say -(1+1)^2=-4 and 3--2=5. Note that I simply can't name a single user input specialized calculator or language that behaves otherwise. I hope this gives an overview on how the common understanding of arithmetic is (outside programming languages, where none exists).

Right now bc is somewhat consistent with some programming languages. I would prefer it to be a arithmetic language in terms of common understanding.
(0003399)
Vincent Lefevre (reporter)
2016-10-07 00:06

 The issue with unary minus is whether it should have a high precedence (as a prefix operator) or it should follow the usual math rules. In C, what was chosen is a high precedence.
(0003400)
Vincent Lefevre (reporter)
2016-10-07 00:15

 Concerning the other issue, you want

echo "3-$var" | bc

to behave like

echo "3-($var)" | bc

when $var is a number. But what about the following?

echo "$var^2" | bc

Currently it behaves like

echo "($var)^2" | bc

if $var is a number, but if you want to change the precedence of the unary minus operator to be less than the power operator, then

echo "$var^2" | bc

will no longer work as you expect: if $var is -2, it would give -4 instead of 4.
(0003401)
Don Cragun (manager)
2016-10-07 02:14

 I find this entire discussion disappointing. If you would like to have a calculator in the standard that performs unambiguous evaluation of adjacent operators with a different precedence than that specified by the standard for the bc utility; please design a new utility that behaves the way you want it to behave, implement that utility, show us that applications and/or users are using your new utility in preference to bc, and submit that new utility for standardization.

Please DO NOT suggest that all of the code that is using bc today should be modified to meet a new, incompatible definition of how bc should behave. The language bc recognizes for arithmetic evaluations was created over 35 years ago. Now is NOT the time to change that language. Yes it is possible to send input to bc expressions that might not be evaluated in an obvious way if you don't pay attention to the current specification of the bc utility; it is also possible to use space between operators and parentheses around expressions to properly evaluate any calculation you want to perform.
(0003403)
tobiasm (reporter)
2016-10-07 22:44

 Exactly, echo "$var^2" will return -4. My suggestion does not take care of this, because there is no way of implementing common arithmetic and getting rid of this issue. The user still needs to know that shell substitutes the variable before passing the whole string (no special knowledge about bc's behaviour required however, which is the great plus). One needs to know the same thing about shell right now (see the other example), so you're right in this point there would be no improvement.
"3-$var" was mainly used to show that -- will occur when obviously meant as subtraction.



Don't get me wrong, this is no bug report or something similar. bc is ok right now even without change, but I would not oppose improvements for the sake of persistence.
35 years ago computers weren't widespread enough for a common understanding (of how arithmetic user input should be calculated) to become explicit. As I hopefully pointed out, this has changed. So I propose bc to be changed.
(0003942)
kre (reporter)
2018-03-28 03:08

 I agree with note 3401. Please simply close this issue, this is not
the place to invent new languages (which is what is being asked here).
bc has been around a very long time, warts and all, changes to it are
inappropriate - on the other hand, if the specification could be clarified
to make it clearer that it acts the way it does, that would be a reasonable
request to make.
(0003949)
Don Cragun (manager)
2018-04-12 15:11

 After discussing this bug on the 2018-04-12 conference call, we decided to reject this issue for the reasons stated in Note: 0003401.

- Issue History
Date Modified Username Field Change
2016-10-05 23:40 tobiasm New Issue
2016-10-05 23:40 tobiasm Status New => Under Review
2016-10-05 23:40 tobiasm Assigned To => ajosey
2016-10-05 23:40 tobiasm Name => Tobias Martens
2016-10-05 23:40 tobiasm Section => bc
2016-10-05 23:40 tobiasm Page Number => -
2016-10-05 23:40 tobiasm Line Number => -
2016-10-06 07:41 Vincent Lefevre Note Added: 0003395
2016-10-06 07:41 Vincent Lefevre Note Edited: 0003395
2016-10-06 07:42 Vincent Lefevre Note Edited: 0003395
2016-10-06 08:02 Vincent Lefevre Note Edited: 0003395
2016-10-06 08:09 Vincent Lefevre Note Added: 0003396
2016-10-06 10:19 stephane Note Added: 0003397
2016-10-06 10:21 stephane Note Edited: 0003397
2016-10-06 10:39 stephane Note Edited: 0003397
2016-10-06 23:05 tobiasm Note Added: 0003398
2016-10-07 00:06 Vincent Lefevre Note Added: 0003399
2016-10-07 00:15 Vincent Lefevre Note Added: 0003400
2016-10-07 02:14 Don Cragun Note Added: 0003401
2016-10-07 22:44 tobiasm Note Added: 0003403
2018-03-28 03:08 kre Note Added: 0003942
2018-04-12 15:11 Don Cragun Note Added: 0003949
2018-04-12 15:12 Don Cragun Interp Status => ---
2018-04-12 15:12 Don Cragun Status Under Review => Closed
2018-04-12 15:12 Don Cragun Resolution Open => Rejected

Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker