In Python, how I do use subprocess instead of os.system?

I have a Python script that calls an executable program with various arguments (in this example, it is 'sqlpubwiz.exe' which is the "Microsoft SQL Server Database Publishing Wizard"):

import os

sqlpubwiz = r'"C:\Program Files\Microsoft SQL Server\90\Tools\Publishing\sqlpubwiz.exe"'
server = 'myLocalServer'
database = 'myLocalDatabase'
connection_values = ['server=' + server, 'database=' + database, 'trusted_connection=true']
connection_string = ';'.join(connection_values)
dbms_version = '2000'
sqlscript_filename = 'CreateSchema.sql'

args = [
        sqlpubwiz,
        'script',
        '-C ' + connection_string,
        sqlscript_filename,
        '-schemaonly',
        '-targetserver ' + dbms_version,
        '-f',
]

cmd = ' '.join(args)
os.system(cmd)

This code runs properly but I have would like to get into the habit of using subprocess since it is intended to replace os.system. However, after a few failed attempts, I can not seem to get it work properly.

How would the above code look like if it was converted to use subprocess in place of os.system?

Asked by: Chelsea703 | Posted: 28-01-2022

Answer 1

import subprocess
p=subprocess.Popen(args, stdout=subprocess.PIPE)
print p.communicate()[0]

It would look pretty much the same. But the path should not be r'"whatever the path is"'. Because that gives me an error. You want "the path with escaped backslashes" or r'the path without escaping'.

Also args should be of the form ['-arg', 'args'] instead of ['arg argsval'].

Answered by: Miranda635 | Posted: 01-03-2022

Answer 2

Remove quotes from the name of the executable. On the first line of your example, instead of

sqlpubwiz = r'"C:\Program Files\Microsoft SQL Server\90\Tools\Publishing\sqlpubwiz.exe"'

use:

sqlpubwiz = r'C:\Program Files\Microsoft SQL Server\90\Tools\Publishing\sqlpubwiz.exe'

That's because you don't have to escape anything since a shell won't be involved.

Then just use subprocess.call(args) (don't join the args, pass them as a list)

If you want to capture the output (os.system can't do it) just follow subprocess documentation:

result = subprocess.Popen(args, stdout=subprocess.PIPE).communicate()[0]
print result

Answered by: Chester994 | Posted: 01-03-2022

Answer 3

Below is my revised code based on Carlos Rendon (and nosklo) help and suggestions:

# import os
import subprocess    

sqlpubwiz = r'C:\Program Files\Microsoft SQL Server\90\Tools\Publishing\sqlpubwiz.exe'
server = 'myLocalServer'
database = 'myLocalDatabase'
connection_values = ['server=' + server, 'database=' + database, 'trusted_connection=true']
connection_string = ';'.join(connection_values)
dbms_version = '2000'
sqlscript_filename = 'CreateSchema.sql'       

args = [
            sqlpubwiz,
            'script',
            '-C',
            connection_string,
            sqlscript_filename,
            '-schemaonly',
            '-targetserver',
            dbms_version,
            '-f',
    ]   

# cmd = ' '.join(args)
# os.system(cmd)

subprocess.call(args)

(Note: The original argument values that contained spaces needed to be converted into separate list items.)

Answered by: Ned624 | Posted: 01-03-2022

Answer 4

FYI, subprocess has a list2cmdline() function that will let you see the string that Popen will be using.

Your version gives:

'"C:\\Program Files\\Microsoft SQL Server\\90\\Tools\\Publishing\\sqlpubwiz.exe" script "-C server=myLocalServer;database=myLocalDatabase;trusted_connection=true" CreateSchema.sql -schemaonly "-targetserver 2000" -f'

with extra quotes around "-C server=myLocalServer;database=myLocalDatabase;trusted_connection=true" and "-targetserver 2000".

Properly formatted:

args = [
        sqlpubwiz,
        'script',
        '-C', connection_string,
        sqlscript_filename,
        '-schemaonly',
        '-targetserver', dbms_version,
        '-f',
]

gives:

'"C:\\Program Files\\Microsoft SQL Server\\90\\Tools\\Publishing\\sqlpubwiz.exe" script -C server=myLocalServer;database=myLocalDatabase;trusted_connection=true CreateSchema.sql -schemaonly -targetserver 2000 -f'

Also, minor point, but it's a good habit to make sequences such as args that don't need to be mutable into tuples instead of lists.

Answered by: Cadie927 | Posted: 01-03-2022

Answer 5

Please remember that os.system uses the shell, and so you must really pass

shell=True

to the Popen constructor/call to emulate it properly. You may not actually need a shell, of course, but there it is.

Answered by: Brooke520 | Posted: 01-03-2022

Answer 6

This isnt an answer directly to your question but I thought it might be helpful.

In case you ever want more granular control over what is returned for exception handling etc, you can also check out pexpect. I have used it in situations where the process I was calling didn't necessarily exit with normal status signals, or I wanted to interact with it more. It's a pretty handy function.

Answered by: Tess404 | Posted: 01-03-2022

Answer 7

Windows commands will accept forward slashes '/' in place of backslashes in pathnames, so you can use the former to avoid escaping backslashes in your command strings. Not exactly an answer to your question, but perhaps useful to know.

Answered by: Grace468 | Posted: 01-03-2022

Similar questions

subprocess - Python os.system without the output

I'm running this: os.system("/etc/init.d/apache2 restart") It restarts the webserver, as it should, and like it would if I had run the command directly from the terminal, it outputs this: * Restarting web server apache2 ... waiting [ OK ] Ho...

python - os.system or subprocess to pass command to shell

I am trying to execute a command in a shell and pipe the output at the same time for filtering. the relevent code looks like: import os n=raw_input("enter cmd") os.system(n + ' | grep x') the result is Syntax error: Redirection unexpected. It is on ubuntu, seemed like some references online mentioned this, but none I could directly associate. Seems...

Using Subprocess or os.system to run shell commands in python

I need to write a code in Python using functions that a friend of mine developed in shell. Is that possible? Can I do something like output = subprocess.call('friends_developed_function', shell = True)

run os.system command as subprocess python

I want to run the following os.system command as a subprocess in order to know its pid since I want to be able to perform a proc.terminate() Bellow is the os.system() Command that works. Note that the bellow command already opens a subshell since thats what the setview is doing, and then executes a python script inside that subshell and then exit...

Execute Jar in Python subprocess vs. os.system

I am learning how to exec a jar in python. I found two ways, 1) subprocess and 2) os.system. In my code, only os.system works and am I wondering why. Here is my code. import os os.chdir('/Users/Jim/Desktop') import subprocess from subprocess import Popen, PIPE, STDOUT p = Popen(['java','-cp','dfp-api-1.0-SNAPSHOT-standalone.jar', \ 'dfp.axis.v201611.lineitemservices.GetAllLineItems', ...

linux - python - i can't get the same result from os.system in subprocess

i want to convert from os.system into subprocess the following os.system("egrep 'Invalid user' /home/bits/Desktop/Assessment/auth.log | cut -d ' ' -f 11 >tempfile.txt ") it works but when i write the same statement using subprocess it will create an empty file , does some1 know why and how to resolve that ? thx a1=subprocess.Popen(["egrep \"Invalid users\" /home/bits/De...

Call external command from python without using neither os.system nor subprocess

Is there any other way to call external command from python without using neither os.system nor subprocess. This is related to my previous question ununderstandable behavior subprocess.Popen(cmd,stdout) and os.system(cmd) The problem is that os.system...

python - subprocess and os.system do not work when launched from flask

I need to run a command using python code and I tried to use both os.system and subprocess however both didn't work for some reason. Here's my code: @app.route('/run-script') def run_script(): subprocess.call('python3.6 GoReport.py --id 31-33 --format word', cwd="working_dir", shell=True) return flask.render_template('results.html', **locals()) Running this command from terminal dire...

subprocess - Python: adding args to os.system

So I am importing the "args" used on a console project and trying to add them to the cmd line but am getting "can only concatenate str (not "bytes") to str". I pull the answers to the args from the microprocessors responses which I list below: level = 5 name = T7 seed = b'cd663319' import subprocess string="ConsoleUnlock.exe --database=database.json" + " -...

python - Is there a way to capture the output of a os.system / subprocess function WHILE RUNNING?

This question already has answers here:

Prevent Python subprocess from passing fds on Windows?

Python's subprocess module by default passes all open file descriptors to any child processes it spawns. This means that if the parent process is listening on a port, and is killed, it cannot restart and begin listening again (even using SO_REUSEADDR) because the child is still in possession of that descriptor. I have no control over the child process. The subprocess POpen constructor does accept a close_fds argu...

Use Python 2.6 subprocess module in Python 2.5

I would like to use Python 2.6's version of subprocess, because it allows the Popen.terminate() function, but I'm stuck with Python 2.5. Is there some reasonably clean way to use the newer version of the module in my 2.5 code? Some sort of from __future__ import subprocess_module?

python - How to properly interact with a process using subprocess module

I'm having problems redirecting stdio of another program using subprocess module. Just reading from stdout results in hanging, and Popen.communicate() works but it closes pipes after reading/writing. What's the easiest way to implement this? I was playing around with this on windows: import subprocess proc = subprocess.Popen('python -c "while True: print \'Hi %s!\' % raw_input()"', ...

pipe - Python subprocess "object has no attribute 'fileno'" error

This code generates "AttributeError: 'Popen' object has no attribute 'fileno'" when run with Python 2.5.1 Code: def get_blame(filename): proc = [] proc.append(Popen(['svn', 'blame', shellquote(filename)], stdout=PIPE)) proc.append(Popen(['tr', '-s', r"'\040'"], stdin=proc[-1]), stdout=PIPE) proc.append(Popen(['tr', r"'\040'", r"';'"], stdin=proc[-1]), stdout=PIPE) proc.append(P...

python - What does the 'shell' argument in subprocess mean on Windows?

The docs for the subprocess module state that 'If shell is True, the specified command will be executed through the shell'. What does this mean in practice, on a Windows OS?

python - subprocess module: using the call method with tempfile objects

I have created temporary named files, with the tempfile libraries NamedTemporaryFile method. I have written to them flushed the buffers, and I have not closed them (or else they might go away) I am trying to use the subprocess module to call some shell commands using these generated files. subprocess.call('cat %s' % f.name) always fails saying that the named temporary file does not...

loops - Python and subprocess

This is for a script I'm working on. It's supposed to run an .exe file for the loop below. (By the way not sure if it's visible but for el in ('90','52.6223',...) is outside the loop and makes a nested loop with the rest) I'm not sure if the ordering is correct or what not. Also when the .exe file is ran, it spits some stuff out and I need a certain line printed to the screen (hence where you see AspecificLinfe= ... ). Any...

Python - Subprocess - How to call a Piped command in Windows?

How do I run this command with subprocess? I tried: proc = subprocess.Popen( '''ECHO bosco|"C:\Program Files\GNU\GnuPG\gpg.exe" --batch --passphrase-fd 0 --output "c:\docume~1\usi\locals~1\temp\tmptlbxka.txt" --decrypt "test.txt.gpg"''', stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, ) stdout_value, stderr_value = proc.communicate() but got:...

Can I send SIGINT to a Python subprocess on Windows?

I've got a Python script managing a gdb process on Windows, and I need to be able to send a SIGINT to the spawned process in order to halt the target process (managed by gdb) It appears that there is only SIGTERM available in Win32, but clearly if I run gdb from the console and Ctrl+C, it thinks it's receiving a SIGINT. Is there a way I can fake this such that the functionality is available on all platforms?...

In Python 2.5, how do I kill a subprocess?

I am using the subprocess package in Python to run a subprocess, which I later need to kill. However, the documentation of the subprocess package states that the terminate() function is only available from 2.6 We are running Linux with 2.5 and for backwards compatibility reasons I cannot upgrade to 2.6, what is the alternative? I am guessing that these functions are convenience methods for something.

Still can't find your answer? Check out these communities...

top