skeleton.py not added even though --no-skeleton wasn't specified

[sgbaird]

Description of your problem

Please provide a minimal, self-contained, and reproducible example.

conda create -n pyscaffold-bug python==3.9.*
conda activate pyscaffold-bug
pip install git+https://github.com/pyscaffold/pyscaffold.git git+https://github.com/pyscaffold/pyscaffoldext-dsproject.git git+https://github.com/pyscaffold/pyscaffoldext-markdown.git

putup my_project --github-actions --markdown --dsproj

Please provide the full traceback using the --very-verbose flag.

(mp-time-split) PS C:\Users\sterg\Documents\GitHub\sparks-baird> putup my_project --github-actions --markdown --dsproj --very-verbose                    
    activate  pyscaffold.extensions.github_actions
    activate  pyscaffold.extensions.no_skeleton        
    activate  pyscaffold.extensions.pre_commit
    activate  pyscaffoldext.dsproject.extension        
    activate  pyscaffoldext.markdown.extension
      invoke  pyscaffold.actions:get_default_options   
       check    is git installed...
         run    git.exe --version
         run    git.exe --version
       check    is git configured...
         run    git.exe config --get user.name
         run    git.exe config --get user.email
         run    git.exe config --get user.name
         run    git.exe config --get user.email
      invoke  pyscaffold.actions:verify_options_consistency
      invoke  pyscaffold.structure:define_structure    
      invoke  pyscaffoldext.markdown.extension:add_doc_requirements
      invoke  pyscaffoldext.dsproject.extension:add_dsproject
      invoke  pyscaffold.extensions.pre_commit:find_executable
      invoke  pyscaffold.extensions.pre_commit:add_files
      invoke  pyscaffold.extensions.no_skeleton:remove_files
      invoke  pyscaffold.extensions.github_actions:add_files
      invoke  pyscaffoldext.markdown.extension:replace_files
      invoke  pyscaffoldext.dsproject.extension:replace_readme
      invoke  pyscaffold.actions:verify_project_dir    
      verify    does project path my_project exist...  
       check    is . already a git repo...
       chdir    .
         run      git.exe rev-parse --git-dir
        info      last command failed with "fatal: not a git repository (or any of the parent directories): .git"
      invoke  pyscaffold.update:version_migration      
      invoke  pyscaffold.structure:create_structure    
      create    my_project
      create    my_project\.gitignore
      create    my_project\.coveragerc
      create    my_project\.readthedocs.yml
      create    my_project\pyproject.toml
      create    my_project\setup.py
      create    my_project\setup.cfg
      create    my_project\tox.ini
      create    my_project\LICENSE.txt
      create    my_project\src
      create    my_project\src\my_project
      create    my_project\src\my_project\__init__.py
      create    my_project\tests
      create    my_project\tests\conftest.py
      create    my_project\docs
      create    my_project\docs\conf.py
      create    my_project\docs\Makefile
      create    my_project\docs\_static
      create    my_project\docs\_static\.gitignore     
      create    my_project\docs\requirements.txt       
      create    my_project\docs\index.md
      create    my_project\docs\readme.md
      create    my_project\docs\license.md
      create    my_project\docs\authors.md
      create    my_project\docs\changelog.md
      create    my_project\docs\contributing.md        
      create    my_project\configs
      create    my_project\configs\.gitignore
      create    my_project\data
      create    my_project\data\.gitignore
      create    my_project\data\external
      create    my_project\data\external\.gitignore    
      create    my_project\data\interim
      create    my_project\data\interim\.gitignore     
      create    my_project\data\preprocessed
      create    my_project\data\preprocessed\.gitignore
      create    my_project\data\raw
      create    my_project\data\raw\.gitignore
      create    my_project\Dockerfile
      create    my_project\environment.yml
      create    my_project\models
      create    my_project\models\.gitignore
      create    my_project\notebooks
      create    my_project\notebooks\template.ipynb    
      create    my_project\references
      create    my_project\references\.gitignore       
      create    my_project\reports
      create    my_project\reports\figures
      create    my_project\reports\figures\.gitignore  
      create    my_project\scripts
      create    my_project\scripts\train_model.py      
   chmod 766    my_project\scripts\train_model.py
      create    my_project\.pre-commit-config.yaml     
      create    my_project\.isort.cfg
      create    my_project\.github
      create    my_project\.github\workflows
      create    my_project\.github\workflows\ci.yml    
      create    my_project\README.md
      create    my_project\AUTHORS.md
      create    my_project\CHANGELOG.md
      create    my_project\CONTRIBUTING.md
      invoke  pyscaffold.actions:init_git
       check    is initialization of the git repository my_project needed...
       check    is my_project already a git repo...    
       chdir    my_project
         run      git.exe rev-parse --git-dir
        info      last command failed with "fatal: not a git repository (or any of the parent directories): .git"
  initialize    git repo in my_project...
       chdir    my_project
         run      git.exe init
         run      git.exe add .gitignore
         run      git.exe add .coveragerc
         run      git.exe add .readthedocs.yml
         run      git.exe add pyproject.toml
         run      git.exe add setup.py
         run      git.exe add setup.cfg
         run      git.exe add tox.ini
         run      git.exe add LICENSE.txt
         run      git.exe add src\my_project\__init__.py
         run      git.exe add tests\conftest.py
         run      git.exe add docs\conf.py
         run      git.exe add docs\Makefile
         run      git.exe add docs\_static\.gitignore
         run      git.exe add docs\requirements.txt
         run      git.exe add docs\index.md
         run      git.exe add docs\readme.md
         run      git.exe add docs\license.md
         run      git.exe add docs\authors.md
         run      git.exe add docs\changelog.md
         run      git.exe add docs\contributing.md
         run      git.exe add configs\.gitignore
         run      git.exe add data\.gitignore
         run      git.exe add data\external\.gitignore
         run      git.exe add data\interim\.gitignore
         run      git.exe add data\preprocessed\.gitignore
         run      git.exe add data\raw\.gitignore
         run      git.exe add Dockerfile
         run      git.exe add environment.yml
         run      git.exe add models\.gitignore
         run      git.exe add notebooks\template.ipynb
         run      git.exe add references\.gitignore
         run      git.exe add reports\figures\.gitignore
         run      git.exe add scripts\train_model.py
         run      git.exe add .pre-commit-config.yaml
         run      git.exe add .isort.cfg
         run      git.exe add .github\workflows\ci.yml
         run      git.exe add README.md
         run      git.exe add AUTHORS.md
         run      git.exe add CHANGELOG.md
         run      git.exe add CONTRIBUTING.md
         run      git.exe commit -m "Initial commit"
      invoke  pyscaffold.extensions.pre_commit:install
       chdir    my_project
         run      ..\..\..\..\Miniconda3\scripts\pre-commit.EXE install
  
A pre-commit hook was installed in your repo.
It is a good idea to update the hooks to the latest version:

    pre-commit autoupdate

Don't forget to tell your contributors to also install and use pre-commit.

      invoke  pyscaffold.actions:report_done
done! 🐍 🌟 ✨
(mp-time-split) PS C:\Users\sterg\Documents\GitHub\sparks-baird>

Please provide any additional information below.

Workaround was to copy https://github.com/pyscaffold/pyscaffold/blob/master/src/pyscaffold/templates/skeleton.template into my repo. Wasn't sure if there was an easy way to bypass the --no-skeleton extension being activated.

Versions and main components

• PyScaffold Version: 4.2.2
• Python Version: 3.9.12
• Operating system: Windows
• How did you install PyScaffold: (conda/pip)

[FlorianWilhelm]

Hi @sgbaird, thanks for posting this issue. Actually it's an intended behaviour as --dsproject activates --no-skeleton by default. The idea was that in case of a data science project the skeleton.py file might not be that useful but I agree that it is a bit counterintuitive. Which part of this file did you need and what is your use-case?

[sgbaird]

@FlorianWilhelm apologies for the delay. I've been renaming skeleton.py to core.py and using it in full (i.e. adding the main forward-facing class, adapting the CLI). I've also noticed the generation of train_model.py which I suppose I could use in place of skeleton.py and simply move it into src/my_package so that it's available after installation by a user of my_package via pip or conda. I'm not sure I'm a big fan of the use of config files as in train_model.py for my packages (not very common in materials informatics packages), though of course I could just remove that option.

In particular, I've been exposing some basic functionality of the packages via CLI in https://github.com/sparks-baird/xtal2png#command-line-interface-cli and starting up the same in https://github.com/sparks-baird/mp-time-split.

The idea is someone can:

pip install xtal2png

and then use the entry_point in a terminal if they want to:

xtal2png --decode ...

Though using the Python functions directly would generally be preferable/recommended.

[FlorianWilhelm]

Thanks @sgbaird for explaining your use-case. I should really rethink the automatic --no-skeleton flag in --dsproject as you showed me that an example for cli interfaces could be helpful also for Data Science projects.

So far in data science projects I saw mostly that people work in their git checkout within the directory structure of the repo. The use the notebooks, reference data in data/* and so on. Still they have the advantage of a proper Python package with the help of PyScaffold.

Let's add an issue to dsproject to include the skeleton again and track this issue there.