Posts

Showing posts with the label Dev

Python short course: Thoughts about data structure

The data structure is the way to store data on the computer efficiently. There are various methods for this. In a nutshell, we focus on space-how to store, and time-how is efficient to store.

Python short course: Big O for algorithm analysis

The big O is an asymptotic sense for the algorithm analysis.  $$f(n) \leq cg(n)~~~n\geq n_0$$

Dev tip: read tech doc to speed up develop

 During the development, some tips to speed up the process: It's crucial to read the documentation of one platform, one package, one tool etc. Understand that we can search lots of helpful docs online, but usually, the direct and original one might be the better one. First, to go to the functions/methods in the docs fast and to read the arguments, then the requirement of them, also what will be returned by the function. Especially, keep an eye on the default parameters. Some are good, some are not and need to change.

Dev tip: Chunk the data as a good practice to reduce memory use

 Chunk the data as a good practice to reduce memory use.

Python short course: A good practice to build dataframe for new data and columns

 During the data science (DS) projects, often we need to build and extract features to put them into a new column in dataframe. A good practice is to get them into a dict first, then put it into list. Finally, use pd.DataFrame() to convert them into dataframe. 

Dev tip: Git remove and delete staging file

 When you wanna remove the staging file, please use git rm --cached file_name This will stop tracking the staging file and also will delete the remote file uploaded.

Python short course: Convert String List to List

We can use the method below to convert a string list to list. import ast string_list = "['hello', 'world']" ast.literal_eval(string_list)

Writing using Latex: remove the first page number in Latex

 Sometimes, you may want to leave the first page number to empty. Then you can use \thispagestyle{empty}, put it after \maketitlet The link is  remove page number of first page

Dev tip: set terminal:integrated:Inherit Env to False to let the terminal work well for virtual env

Image
 When you use vscode terminal, sometimes you may meet that the terminal does not work well with virtual environment, such as env and conda. The terminal may walk into the env automatically by choosing the right python interpreter. However, the python is inherited from root, not from the virtual env. So by the reminder of vscode itself, we could set terminal:integrated:Inherit Env to False to avoid the issue. terminal setting

Dev tip: add 'keychain' to solve Git: git@gitlab.com: Permission denied (publickey) with VSC

Image
 If you set a password to push the commit to github or gitlab, it's safer. However, this will cause the issue: Git: git@gitlab.com: Permission denied (publickey). A solution is to regenerate the ssh key without password. But a better one is to add 'keychain' to ssh and solve this. It's perfect. Thanks Helge's solution. Here is the link to solution. solution and a screen shot is here. add keychain

Dev tip: Avoid the same variable name in notebook

 In practical, it is easy and convenient to keep some variable name same. Some specific names are used all the same. But this will bring some trouble later when you debug or rerun some cell. The variables are confused and display the latest value. It is annoying. So for a safe use, it is better to use different name for variable names, even some waterfall style processing.