Potential AV on TPythonEngine.DoOpenDll (Linux)

[tomekkosinski]

DoOpenDl loop for all PYTHON_KNOWN_VERSIONS defined versions. But if the version is not found then inherited is called with an empty name which causes AV. Inherited should be called only if UseLastKnownVersion is false.

`procedure TPythonEngine.DoOpenDll(const aDllName : string);
var
i : Integer;
begin
if UseLastKnownVersion then
for i:= Integer(COMPILED_FOR_PYTHON_VERSION_INDEX) downto 1 do
begin
RegVersion := PYTHON_KNOWN_VERSIONS[i].RegVersion;
inherited DoOpenDll(PYTHON_KNOWN_VERSIONS[i].DllName);
if IsHandleValid then
begin
DllName := PYTHON_KNOWN_VERSIONS[i].DllName;
APIVersion := PYTHON_KNOWN_VERSIONS[i].APIVersion;
Exit;
end;
end
else begin
RegVersion := SysVersionFromDLLName(aDllName);
inherited; <--- HERE
end;
end;'

[pyscripter]

In Linux I think you have to specify the DllName and DllPath as well as set the UseLastKnownVersion to False. The auto-detection unfortunately does not work.

e.g.
DLLName=libpython3.6m.so
DLLPath=/usr/lib/x86_64-linux-gnu/
(from https://en.delphipraxis.net/topic/4701-create-delphi4python-at-run-time-extend-demo-sample-34/)

but these depend on the Linux distribution.

Same is true for MacOS

// PythonEngine.DllName := 'libpython3.7.dylib';
// PythonEngine.DllPath :=
// '/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/';
// PythonEngine.RegVersion := '3.7';
// PythonEngine.UseLastKnownVersion := False;

from the test files.

[tomekkosinski]

Actually autodetection works for me on Linux and macOS as well. The only problem is that if it fails (eg no libpython installed) then the reported condition raised

[pyscripter]

Good to know that autodetection works in Linux and MacOS.

Where does the AV occur? In SysVersionFromDLLName or in TDynamicDll.DoOpenDll? Could you look at the stack trace of the exception.

DllName is initialized in TPythonInterface.Create. There is no reason to be empty. If UseLastKnownVersion is true then every known version will be tried and then in addition the inherited DoOpenDLL will be called with DLLName, which should do no harm.

Note that if FatalAbort is true and LoadDLL fails the program exits the hard way.

[tomekkosinski]

It seems that empty DLLName is a problem on Linux. I will test again morning.

[pyscripter]

How come it is empty? The constructor adds a default value (the dll name of the latest version).

[tomekkosinski]

I added some trace messages into TDynamicDll.OpenDll and tested on fresh installs

procedure  TDynamicDll.OpenDll(const aDllName : string);
var
  s : string;
begin
  UnloadDll;

  BeforeLoad;

  FDLLHandle := 0;

  WriteLn(ErrOutput, 'name= '+aDllName);
  DoOpenDll( aDllName);

  if not IsHandleValid then begin
    {$IFDEF MSWINDOWS}
    s := Format('Error %d: Could not open Dll "%s"',[GetLastError, DllName]);
    {$ELSE}
    s := Format('Error: Could not open Dll "%s"',[DllName]);
    {$ENDIF}
    if FatalMsgDlg then
      {$IFDEF MSWINDOWS}
      MessageBox( GetActiveWindow, PChar(s), 'Error', MB_TASKMODAL or MB_ICONSTOP );
      {$ELSE}
      WriteLn(ErrOutput, s);
      {$ENDIF}

    if FatalAbort then
      Quit;
  end else
    AfterLoad;
  WriteLn(ErrOutput, 'Success');
end;

Ubuntu 20.04
DllName="libpython3.11.so"
DllName="libpython3.10.so"
DllName="libpython3.9.so"
DllName="libpython3.8.so"
DllName="libpython3.7m.so"
DllName="libpython3.6m.so"
DllName="libpython3.5m.so"
DllName="libpython3.4m.so"
DllName="libpython3.3m.so"
DllName=""
Error: Could not open Dll “”
Python could not be properly initialized. We Must quit.

Ubuntu 18.04
DllName="libpython3.11.so"
DllName="libpython3.10.so"
DllName="libpython3.9.so"
DllName="libpython3.8.so"
DllName="libpython3.7m.so"
DllName="libpython3.6m.so"
DllName="libpython3.5m.so"
DllName="libpython3.4m.so"
Crash!

Ubuntu 20.04 after “sudo apt-get -y install libpython3.8-dev”
Proof that LastKnowVersion works
DllName="libpython3.11.so"
DllName="libpython3.10.so"
DllName="libpython3.9.so"
DllName="libpython3.8.so"
Success

[pyscripter]

I don't get it.

1. OpenDLL should be called just once, by LoadDLL.
2. Your write statement is WriteLn(ErrOutput, 'name= '+aDllName); but in the output I see 'DLLName'. So this comes from another write statement. And I see no output from this statement!
3. Where is DLLName set to an empty string?
4. In which statement does the crash occur?

Is the PythonEngine component DllName property empty?

[pyscripter]

I am thinking of changing TPythonEngine.DoOpenDll to

procedure TPythonEngine.DoOpenDll(const aDllName : string);
var
  i : Integer;
begin
  if UseLastKnownVersion then
    for i:= Integer(COMPILED_FOR_PYTHON_VERSION_INDEX) downto 1 do
    begin
      RegVersion := PYTHON_KNOWN_VERSIONS[i].RegVersion;
      inherited DoOpenDll(PYTHON_KNOWN_VERSIONS[i].DllName);
      if IsHandleValid then
      begin
        DllName := PYTHON_KNOWN_VERSIONS[i].DllName;
        APIVersion := PYTHON_KNOWN_VERSIONS[i].APIVersion;
        Exit;
      end;
    end
  else 
  begin
    RegVersion := SysVersionFromDLLName(aDllName);
    inherited;
  end;
end;

Does this work OK?

[tomekkosinski]

Sorry I copy edited code 'name'->'DllName'. The crash happens in 'inherited' so the output is not recorded

My initial proposal is the same as yours and it fixed the problem.

[pyscripter]

Sorry I copy edited code 'name'->'DllName'. The crash happens in 'inherited' so the output is not recorded

I am still puzzled. See above. OpenDLL should be called only once. Also, is the PythonEngine component DllName property equal to an empty string?

inherited calls TDynamicDll.DoOpenDll cannot you debug your program to see the exact statement causing the crash?

[pyscripter]

I am reluctant to change the code since as it stands, it gives P4D an extra chance of finding a python dll with a DLLName different than then standard names, which could be useful in Linux.

[pyscripter]

I eventually implemented the suggested change.