6 things that I miss in bash
What is the most frequently used piece of software on your computer? Here’s one program that holds one of the highest place in my list of most frequently used programs. I am talking about… bash.
In case you don’t know, this is the program behind large portion of the Linux command line. It is the program that actually makes majority of work turning Linux command line into such a magnificent tool it is.
As for use frequency, on my computer the only program that beats it is perhaps a web browser. I use it more often than mail client, text processor or just any other program.
Now, one thing that entertains me is that there is absolutely no hype around bash. I mean, you probably know that next version of Firefox coming tomorrow (I am writing this on June 9th, 2009). But do you know when next version of bash is coming? Do you even know what is the version of bash that you currently use? Well, you are not alone – I have no clue either 
One thing that concerns me is that bash almost didn’t change for the last ten years. I mean the only new feature that, to my knowledge, it didn’t support ten years ago is bash_completion – that is the advanced version of plain completion that it does to files. I could be wrong about this of course, but one thing is certain. Whatever the changes are, they are unnoticeable.
So, I thought that I’d compile a list of features that I would like to see in the upcoming version of bash. Now really, if only one of them would become a reality, I’d chew my pair of orange Crocs (I do have a pair of orange Crocs), put it on video and post it on youtube. Alright, I won’t do that. But it would still be very nice to see these happening.
The list:
1 – History sharing
I often open multiple shells. It would be really nice to be able to synchronize history between them. Right now, it seems that bash writes history file when it shuts down. If you open another session after history has been saved, you can access commands that you’ve typed in in the previous session. Otherwise, the history between two sessions is completely independent. Not good.
2 – Completion of directory and file names based on their usage
I think that 90% of cd‘s that I do are all destined to the same directory. Why on earth I see the list of possible destination for cd in alphabetical order every time I press TAB?
3 – Completion of make targets
I know this has already been implemented. But I am wondering why it isn’t part of bash_completion package on Ubuntu. OpenSuSE is no better – it does completion only to some yast stuff.
4 – Show a browseable and searchable list of files when hitting TAB key twice
I guess you all know that. Single TAB key completes a word if it can make a single match. Double TAB shows a list of choices for completion. In case the list is too long, it builds a long list and pipes it through more.
What I would like to see is a browseable and searchable list. One that I can browse with arrow keys. I also would like to be able to enter portion of the file name and in that case bash should filter out everything that doesn’t match the criteria I’ve entered.
5 – Completion of scp, on remote host
I guess it happened to all of us. Copying file to a remote computer with scp, is always difficult because you actually have to remember the path on remote computer. Wouldn’t it be great if bash could complete remote paths as well?
Now, I know this is very difficult to implement. Basically you have to spawn ssh with ls command, wait for it to complete and parse the results. Oh and there is a password thing. You gotta take care of password prompt that ssh may produce. Have no idea how to solve this issue. I do see a couple of options, but all of them are far from being ideal.
6 – Fix typos
This one is simple. As long as I don’t hit the TAB key, no need to fix anything. Once I hit the TAB key, shell should check if the command I’ve entered makes sense and if not try to fix it. Good example of what could make sense are file names. vi <filename> probably means that you want to edit a file. If the file does not exist and yet there’s another file with almost identical name, fix it. Same with cd. You don’t cd into directories that don’t exist. So once shell detects an attempt to do so and the user asks for help (by pressing the TAB key), shell can fix the typo and make the user smile.
This is it. I hope to see any of these happening one day. In the meantime, enjoy the list and by all means leave comments with your ideas of bash enhancements.
1. History Sharing:
From my ~/.bashrc:
# don’t put duplicate lines in the history. See bash(1) for more options
export HISTCONTROL=ignoredups
# … and ignore same sucessive entries.
export HISTCONTROL=ignoreboth
shopt -s histappend
export PROMPT_COMMAND=’history -a; history -n; history -w’
From “man bash”: PROMPT_COMMAND If set, the value is executed as a command prior to issuing each primary prompt.
Partial output from “help history”:
`-a’ means to append history lines from this session to the history file.
`-w’ option writes out the current history to the history file
`-n’ means to read all history lines not already read from the history file and append them to the history list.
This will append the current session history to the end of the history file instead of overwriting it and do so before displaying each prompt.
History sharing has been available for quite a while now.
#6 is the famous “dwim” (do what I mean) feature that turned out to be a Very Bad Thing. Even from your example, the problem is obvious. You want to create files foo.h and foo.c. You edit foo.h and save it. Then at the command line you enter “vi foo.c” and the dwim feature changes it to “vi foo.h” because foo.c doesn’t exist but there’s a similar file foo.h that does exist. Bad.
The classic Really Bad dwim example, which I think actually happened many years ago, involved entering “rm -f *.obj” and seeing dwim change it to “rm -f *” because no .obj files were present. Say goodbye to all your files. Fortunately, your idea of dwim seems less aggressive than that.
One thing that would be kind of cool would be if bash refused to execute a command if it was impossible to expand an argument with wildcards. So you enter “rm -f *.obj” and bash simply prints “No files match *.obj” and sets the last command exit status to a non-zero value without running rm at all. I’m sure that behavior would break things, though.
#4 would be nice, but it would depend on having a terminal smart enough to support ncurses. It’s probably almost possible to take that for granted these days, but you never know.
#1 and #2 are both possible, but I would hate them passionately. I often depend on knowing how many up-arrows it will take to get back to some command I used recently. If history is being shared between two shells, forget about that. Similarly, if bash is trying to be “smart” and cycle through completion options in MRU order, it makes the sequence less predictable.
I should have said: export HISTCONTROL=ignoreboth:erasedups
… and eliminated the first HISTCONTROL line.
#5 works with bash_completion if you have an ssh key and agent running. I use it a lot.
1. Some decent looking functions. Is it necessary to write something like just cryptograpers can put up with to get a string length
${#VARIABLE}
when you expect something like
length (variable)
Sure we have to avoid collisions with possible future file names, but it cannot be too hard.
The python shell does it well.
2. Namespaces.
3. Decent mathematical expressions.
4. Possibility to detect misspelled variables, for instance using optional variable declarations, preferrably with type and casts for type conversions.
5. The TCP and UDP hacks a neat, but how do I set up a simple server in bash? I have to use, for instance, awk for that.
6. Local variables is functions.
7. Better builtin string handling. Too often I have to write something like
mv $FILE $(echo $FILE | sed s/XXXX/YYYY/)
in order to rename a bunch of files. It usually works, but always leaves a fear that something will go wrong.
8. Variables that can hold binary data strings. Functions to handle binary data strings.
Binary and text structures, unions.
9. Functions for random access to files;
10. Possibility to read binary data into variables (integers, floats, bits, etc),
say to check a magic number or follow a binary (or ASCII) B+ index to access a data record.
11. System V messaging. Some nice way to access shared memory, semaphores and message queues. These seem to be a bit out of fashion, so why not UNIX domain sockets, dbus access or whatever people prefer these days.
12. CORBA is out of fashion and may be a bit exaggerated to add it.
13. Threads? I am no fan of thread, so just maybe.
14. A way to parse CSV data. (Also awk needs this).
15. Nice tools to read configuration file.
16. XML parsing is not needed. XML is overhyped and mostly unnecessary. Use ASN.1 instead.
17. Random access to compressed files.
Well, that was too much already and some items on the wishlist are surely already there.
Summa summarum, add the possibility to do what you use python for.
Best regards, Raymond
@Raymond
“mv $FILE $(echo $FILE | sed s/XXXX/YYYY/)”
mv $FILE ${FILE/XXX/YYY}
I use it al the time in fors to rename files.
Also this one:
mv $FILE ${FILE: -3}ogg
That will change a file’s suffix (delete last 3 characters) and change it for ogg.
@Scott
Wow. This is a really cool stuff. I didn’t know all that. Thanks for sharing it.
Originally Posted By Craig
I disagree with you. I actually emphasized it.
a. I suggest that bash should go on and fix things only if you ask it to do so by pressing TAB key. Otherwise it should not fix anything.
b. I think we can come up with a number of simple rules that would define how bash should help. This may include changing foo.c to foo.h. This can be many other things as well. In any case it should not guess what I mean. There should be strict mathematical model that tells it exactly what to do.
I am not sure what this is needed for. I mean why do you care the error message came from bash or from rm itself?
#2 is for completion, not for history. As for #1, it seems that bash already supports this, although I didn’t test it myself yet – see first comment to the article.
@Chris Jones
Can you please explain a bit? How can I make it work for me?
@Raymond
I know how you can make all these happen. Here’s the magic command: ‘chsh -s /usr/bin/python’
I have to agree with point 1, thats something that is sometimes frustrating. I’m not sure what problems would be caused with a rolling updated history file.
@alvare
Cool stuff. Thanks for sharing it.
re the scp completion stuff…
Firstly, you need to be using SSH keys instead of passwords.
Secondly, you need to have a local SSH agent running with your key added, this is so that ssh processes can get the key from the agent without having to ask you for the password.
If you use GNOME or another modern desktop then you almost certainly have an agent running, so you just need to create an ssh key (ssh-keygen) and add it to the agent (ssh-add).
Google should be able to find plenty of more verbose guides on this
Hum have you ever tried ZSH ?
(and you won’t come back on bash)
Because it addresses most or all points you have listed …
So you should give it a try
I second the recommendation for zsh. I’ve been using it for years. The original article reads like a list of reasons that I use zsh over bash. I think it does almost all of those things.
BTW, if you ever write scripts, zsh is wonderful. It’s both Bourne Shell and C-Shell compatible (favors Bourne when there’s a conflict) and also implements a ton of it’s own syntax that really makes things easy. You will be doing yourself a big favor if you decide to learn how zsh works. It is by far the best shell for both interactive and scripting tasks.
Use zsh!
I felt the same. Ever since I moved to zsh (5 year ago) I never looked back
I think point 5 is already implemented.
If you set up a passwordless connection it works.
Grtz
On the risk of overstating it, give zsh a try.
It has #3, #4 and #6 nailed, and probably 1 aswell.
I’ve tried to build #2 and quickly learned that my usage was far less predictable than I expected. The files that I access a lot this week are different from what I will be using next week.
The few files and directories I really use a lot are symlinked from my homedir.
PS. #2 it might work if you restrict yourself to a small subset from the history, and only do it if you are very sure. My own experiments gave to many false positives to be usefull. It might also help if you are able to filter out bad commando’s, as my code also learned my typo’s and gleefully completed them the next time, which took more time to undo than was saved in the first place.
#5 As long as you have ssh key setup on the target host tab completion works fine.
Another way to rename files in place is to use the rename command. It seems like there is at least two rename commands that work totally differently, but this works on at least debian and debian-based distributions (for example ubuntu):
$ rename ‘s/from/to/’ *.jpg
Another interesting way to rename files quickly is with the command qmv from renameutils. It opens a text editor with a list of all matching filenames and then you edit the filenames in the text editor. Of course if you are running a capable text editor like vim you can use regexes and other editor features to change the filenames. Change your mind? just exit the file without saving. Its the most useful if you do alias qmv=”qmv -f sc”.
I used to do the $(echo $file | sed) thing too a lot before I learned about these much nicer ways.
Yet another vote for ZSH, I still use bash for all my shell scripting, but ZSH for my interactive shell. It seems like everything you want are interactive features, and z-shell is very rich in interactive features you can use.
For #2, check out cdargs: http://debaday.debian.net/2009/04/05/cdargs-a-browser-for-cd/
Hey children!
Here is the real great shell: Business Shell (BUSH)
http://www.pegasoft.ca/bush.html
Also, thanks for reminding me about zsh; I had nearly forgotten it. There the TCP tricks are possible.
With respect to Python: sure it is pointless to make bash into Python as we already have Python. Howerver, there are some things that have been done right and it may be a good idea to see how it can be transferred to bash.
Happy scripting, Raymond
P.S.
Also rembmer that “script” and “scripture” are not the same thing. The only thing they have in common is that they are not worth getting fanatical about.
@LeeNukes
Apparently:
1) bash is somehow capable of doing this – see first comment to this article.
2) zsh can do this.
@Chris Jones
Oh silly me. This is so simple
Lots of you have recommended zsh. I am definitely going to look at it. Thanks for reminding me of it’s existence
@Raymond
Thanks for the hint. I’ll look into it.
Well, I would also say that zsh does all that. Especially that ncurses menu thing you point out in #4. That being said, after some time, I got tired of the gizmo and went back to bash. Here are some answers :
#1 and #5 can be considered “alsmost solved”
#3 “works here”. I don’t use Ubuntu, but there may be several packages for bash_completion. In any case, make targets are handled.
#2 is a good idea. I’ve written a set of function for “bookmarkin” to work around this problem. Looks like cdargs does it also.
http://wiki.github.com/joelthelion/autojump seems like a good idea : a frequency analysis lets it jump to the most probable place.
#4 exists “as is” in zsh. Most of the time, though, browsing around the list with arrows is not really time-effective. In bash, I have the following line in my .inputrc :
“\M-a”: menu-complete
This means that when I type Alt-a, bash cycles through the possibilities based on what I have entered so far (bonus : inputrc is readline, so all other tools that use it behave that way !).
#6 can be a problem. Bash let’s you decide to have some typo fixing with shopt -s cdspell . If you type cd /tnp, this will get you to /tmp.
@Craig: shopt -s nullglob; echo *.obj
also quite useful in Bash: inline tab completion and magic-space
see: http://codesnippets.joyent.com/posts/show/1690
Pretty good article.
I agree about the 2nd point (completion of directory names based on their usage), it seems really useful and it should be implemented. The other ones are OK too, but I think point 6 (fix typos) would become very dangerous. This can lead to editing files accidentally, files which you wouldn’t want to edit (I’m thinking of files like /etc/apt/sources.list and sources.list~ or xorg.conf~ or hosts.allow, hosts.deny etc). It would be probably a pain to implement too.
@Dan
It seems that 6st point concerns a lot of people. And it will definitely be a pain to implement. Yet, when I think how useful something like this might be, I think it may be worth the effort.
Thx for information.
So you would like to pick first word in each line of input. Have a look at following epexmlas:Input File[neo@techpulp ~]# cat input.txtjoe wants to be a doctorlisa like to learn swimmingmichael learns to rock[neo@techpulp ~]# Example 1:[neo@techpulp ~]# cat input.txt | cut -f1 -d joelisamichael[neo@techpulp ~]#Example 2:[neo@techpulp ~]# cat input.txt | awk {print $1;}’joelisamichael[neo@techpulp ~]# Example 3:[neo@techpulp ~]# cat input.txt | sed -e s/ .*$//g’joelisamichael[neo@techpulp ~]#Like or Dislike: +1