Skip to content
Snippets Groups Projects
Commit 192971df authored by Ulrich Kerzel's avatar Ulrich Kerzel
Browse files

more on decorators, iterators, generators 

add inner/outer function to 'functions'
parent bcb79fbb
Branches
Tags
No related merge requests found
Show whitespace changes
Inline Side-by-side
pythonintro/Functions.ipynb
+ 22
− 0
View file @ 192971df
%% Cell type:markdown id: tags:
# Functions
We could - in principle - write code by adding more and more statements after one another. This might well work and do what we want, but would have quite some disadvantages.
* The code becomes very long very quickly ("Spaghetti - Code")
* We would need to keep track of *a lot* of variables as each variable is valid all the time.
* The only way to re-use code is to copy/paste it.
It works... but it is neither nice, efficient, readable or maintainable.
Functions help us to structure code. Essentially, they assign a block of code to a call of this function.
The general structure is:
```
def my_func(<arguments>):
....
my code goes here
....
return <return value>
```
We notice the following:
* The function is defined using the keyword ```def```
* The name of the function is given by ```my_func```
* We can pass ```<arguments>``` arguments to the function, This is optional.
* Functions can have one or multiple return value(s), indicated by the keyword ```return```. This is optional.
Some programming languages distinguish between functions that return a value ("functions") or not ("procedures"). Python does not and we call them all "functions".
Therefore, the simples function is something like the following:
%% Cell type:code id: tags:
``` python
# define the function
def say_hello():
print("This is a small function that says Hello.")
# call the function
say_hello()
```
%% Output
This is a small function that says Hello.
%% Cell type:markdown id: tags:
This is probably not the most useful function we can think of - but it allows us to see what we can do:
* We can assign a descriptive name to a block of code. The name should give a short indication about what the function does so that when reading the code we can follow what happens
* We can re-use the code. Whenever we want to execute the same block of code, we call the function. This makes it much easier to write understandable and efficient code - and also helps us to maintain the code. Whenever we find a bug, we need to fix it only in this function.
***Best practice***
Functions should only done one thing. This makes them as short, concise and efficient as possible. It also makes them testable (we know what to expect as output), and more maintainable.
While there is an overhead in calling a function, it is *very* small compared to writing inefficient and lengthy code. Do not hesitate to put even a single line of code into a separate function if you think that this will help to make the code easier to read, debug and maintain.
## Function arguments
Functions can have one or more arguments that we pass when calling the function.
%% Cell type:code id: tags:
``` python
def my_sum(x, y):
return x + y
sum = my_sum(3,4)
print('The sum is: {}'.format(sum))
```
%% Output
The sum is: 7.1
%% Cell type:markdown id: tags:
We can make use of the arguments in several ways: Above we pass two numbers and naturally assume that $x=3, y=4$ when we compute the sum. Here, the order of the variables and the arguments matter. The variables are assigned in the order we pass them.
We can also specify this explicity.
%% Cell type:code id: tags:
``` python
def my_sum(x, y):
print('x={}, y={}'.format(x,y))
return x + y
sum = my_sum(y= 4, x=3)
print('The sum is: {}'.format(sum))
```
%% Output
x=3, y=4
The sum is: 7
%% Cell type:markdown id: tags:
Here, we specify the variables directly and can choose any order.
We can also define default arguments, i.e. a value, that is taken when we do not specify the variable.
%% Cell type:code id: tags:
``` python
def my_sum(x=3, y=4):
print('x={}, y={}'.format(x,y))
return x + y
sum = my_sum()
print('The sum is: {}'.format(sum))
sum = my_sum(1)
print('The sum is: {}'.format(sum))
sum = my_sum(2,6)
print('The sum is: {}'.format(sum))
sum = my_sum(x=2)
print('The sum is: {}'.format(sum))
sum = my_sum(x=2, y=1)
print('The sum is: {}'.format(sum))
```
%% Output
x=3, y=4
The sum is: 7
x=1, y=4
The sum is: 5
x=2, y=6
The sum is: 8
x=2, y=4
The sum is: 6
x=2, y=1
The sum is: 3
%% Cell type:markdown id: tags:
There are a (very) few use-cases where you do not know how many arguments a user will pass. We can do this by preceeding the variable name in the argument list with a single ```*```.
***Note***
These arguments are often abbreviated as ```*args```, as we do not know what they will be.
%% Cell type:code id: tags:
``` python
def print_pets(*args):
print(args)
print('dog')
print('dog', 'cat')
print('dog', 'cat', 'hamster')
```
%% Output
dog
dog cat
dog cat hamster
%% Cell type:markdown id: tags:
Sometimes, we might want to use a function with an arbitrary number of arguments and we also do not know what kind of information is going to be passed. We can use the ```**``` construct to accept as many key-value pairs as are provided.
An example could be:
%% Cell type:code id: tags:
``` python
def update_userprofile(name, **profile):
profile['name'] = name
return profile
user_profile = update_userprofile('Reynolds',
first_name = 'Malcom',
rank = 'Captain',
gender = 'male')
print(user_profile)
```
%% Output
{'first_name': 'Malcom', 'rank': 'Captain', 'gender': 'male', 'name': 'Reynolds'}
%% Cell type:markdown id: tags:
As we can see, the ```**profile``` creates a dictionary into which we (implicitly) add all the key-value pairs we pass as arguments to the function.
Inside the function, we can access and change the dictionary as we normally would (e.g. add a new field with key ```name```).
As we can also see, this makes the code less readable and - because we do not know what the user will pass to the function - we also would not know how to check whether this was intentional, etc.
One good use-case for such a construct is that we want to use a generic way to pass options to our code, such as keyword arguments.
In general, when we find ourselves in the situation to need such constructs with ```*``` or ```**```, we should take the opportunity to reflect if we really need this and if we can use simpler approaches to make our code more readable and maintainable.
## Local variables
%% Cell type:code id: tags:
``` python
def my_function(x, y):
i = 0
print('The value of i is {}'.format(i))
return x + y
sum = my_function(3,4)
# uncommenting this wil fail.
# print('The value of i is {}'.format(i))
```
%% Output
The value of i is 0
%% Cell type:markdown id: tags:
So far, all variables we have used in our code, inside or outside loops are always available: Once we have used a variable, it is defined and the Python interpreter keeps it in memory. This is our ***global*** scope.
As our example above shows, variables that we use inside a function are only valid within this function, a ***local*** scope.
This has the benefit that we can define the variables we need inside the function and we do not need to clean up afterwards. Again, this makes the code more readable and maintainable, as we can move blocks of code *together with the relevant variables* into a function.
***Best practice***
When defining a function, do not rely on global variables defined outside the scope of this function. Instead, all variables that you need inside the function should be passed as arguments, all variables that you need back, should be part of the ```return``` statement.
The following works but is clearly not a good idea:
%% Cell type:code id: tags:
``` python
sum = 0
x = 3
y = 7
def stupid_sum():
global sum
sum = x + y
stupid_sum()
print('The sum is {}'.format(sum))
```
%% Output
The sum is 10
%% Cell type:markdown id: tags:
Here, we use the keyword ```global``` to force Python to make use of the variable ```sum``` defined outside the function such that it modifies this variable (otherwise, a local variable would be created with the same name, but not returned. ---try it by commenting out line with ```global sum```).
Obviously, the keyword exists because there may be some legitimate use-case for it.
Generally, avoid such a pattern and have a clearly defined list of input and output values. Otherwise you loose all benefits to writing a function in the place.
%% Cell type:markdown id: tags:
### Exercise
Rewrite the Fibonacci series as a function. The input value should be the number of Fibonacci numbers generated, the output should be a list containing the Fibonacci numbers.
Call your function and have it return 10 Fibonacci numbers.
The output should be like: ```The Fibonacci numbers are: [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]```
%% Cell type:code id: tags:
``` python
# ... your code here ....
```
%% Cell type:markdown id: tags:
## Functions within functions
We can also define functions within functions, these are called *inner functions*. This can be helpful if we need to define a function that makes sense only within the context of the function we currently define.
The general syntax is:
```
def my_function(argument_1, argument_2,....):
def inner_function_1(argument_X, argument_Y,...):
# code for inner_function_1
def inner_function_2(argument_A, argument_B, ...):
# code for inner_function_2
# code for outer function my_function
```
%% Cell type:markdown id: tags:
# Lambda functions
Lambda functions are small anonymous functions. They can take any number of arguments but can have only one expression.
The term "anonymous" function means that lambda construct effectively defines a function (as we did above) --- but we do not assign a name it to but call it directly.
The general syntax is:
```
lambda arguments : expression
````
In some cases, they can help to write code quite concise and are especially helpful if we use them inside other functions.
We could write our example calculating the sum as:
%% Cell type:code id: tags:
``` python
sum = lambda x, y : x + y
print('The sum is: {}'.format(sum(3,4)))
```
%% Output
The sum is: 7
......
pythonintro/IteratorGeneratorDecorator.ipynb
+ 204
− 1
View file @ 192971df
%% Cell type:markdown id: tags:
# Iterators
We have previously seen that we can iterate over items in, for example, a dictionary or a list.
For example:
%% Cell type:code id: tags:
``` python
my_list = ['cat', 'dog', 'bird']
for item in my_list:
print(item)
```
%% Output
cat
dog
bird
%% Cell type:markdown id: tags:
This loop iterates over all elements in the list, gives us access to each element in turn and then stops once we reach the end of the list.
We could do this manually by creating an *iterator* and then use this to traverse the list, until no more elements are left over and an ```StopIteration``` exception is raised.
We define the iterator with the keyword ```my_iter = iter(my_object)``` and then proceed to the next item with ```next(my_iter)```.
%% Cell type:code id: tags:
``` python
my_list = ['cat', 'dog', 'bird']
my_iter = iter(my_list)
try:
while True:
print(next(my_iter))
except StopIteration as e:
print('Reached the end of the list {}'.format(e))
```
%% Output
cat
dog
bird
Reached the end of the list
%% Cell type:markdown id: tags:
Obviously, this is quite a horrible way to loop over a list (though some programming languages are not far off from doing this as the default way...)
One situation where we need to think about iterators is when we need to define a class that we can iterate over.
Then wen need to implement the "magic" (or dunder) functions ```__iter__()``` and ```__next__()```. Remember that the double underscores before and after the keywords indicate that we should not call these methods ourselves. \
For example, you may need to develop a more fancy list, a counter, an even more powerful dictionary, ...
%% Cell type:code id: tags:
``` python
class Counter:
def __init__(self, start, stop):
# number: our counter we want to iterate over
self.number = start
self.stop = stop
def __iter__(self):
return self
def __next__(self):
# check if we have reached the largest number we want to run over
if self.number > self.stop:
raise StopIteration
else:
current_number = self.number
self.number = self.number + 1
return current_number
my_counter = Counter(0, 5)
for counter in my_counter:
print('Value of the counter is now: {}'.format(counter))
```
%% Output
Value of the counter is now: 0
Value of the counter is now: 1
Value of the counter is now: 2
Value of the counter is now: 3
Value of the counter is now: 4
Value of the counter is now: 5
%% Cell type:markdown id: tags:
# Generators
Generators in python are special functions that remember their state each time you call them. Remeber that with the "normal" function we have seen earlier, we call the function, do our computations or other actions, maybe define local variables, etc. We may return the result of what the function does - but then the fuction is "forgotten".
In some cases, we may want to remember the state of the function. For example, we could read from a very large file: If it's very large, we cannot keep it in memory, but need to parse the content one line at the time. However, it would be very cumbersome to open a file, read one line, remember which line we have read, close the file, open the file, jump to the appropriate place, etc. \
Another use-case is we want to compute a long list of elements - but we do not know how many to start with. We could do this with a ```for``` or ```while``` loop (as we have done previously) and define the start and stop conditions.
Generators provide a neat way to do this without having to decide on start or stop condition in the function and, instead, focus on the function itself.
Generators are defined very similarly to normal functions, the main difference is the keyword ```yield```. When we encounter the ```yield``` statement, the execution of the function is stopped, we return a generator object (instead of the value), and the current state of the function is kept
**Example** \
Infinite sum - we want to obtain the sum of all integers until we decide to stop.
%% Cell type:code id: tags:
``` python
def my_infinite_sum():
sum = 0
while True:
yield sum
sum = sum + 1
# Then we can use this and print a few elements
my_generator = my_infinite_sum()
print(next(my_generator))
print(next(my_generator))
print(next(my_generator))
# we can also loop over this
for i in range(0, 5):
print(next(my_generator))
```
%% Output
0
1
2
3
4
5
6
7
%% Cell type:markdown id: tags:
We can also introduce a stop criterion that prevents us from reaching the ```yield``` statement. Then, as with the iterators above, a ```StopIteration``` exception to signal the end.
%% Cell type:code id: tags:
``` python
def my_infinite_sum(stop):
sum = 0
while sum < stop:
yield sum
sum = sum + 1
my_generator = my_infinite_sum(5)
for i in my_generator:
print(i)
```
%% Output
0
1
2
3
4
%% Cell type:markdown id: tags:
***Exercise***
Rewrite the function for the Fibonacci series as a generator.
%% Cell type:code id: tags:
``` python
def fibonacci():
# ... your code here ...
my_generator = fibonacci()
i = 0
while i < 10:
print('Next Fibonacci number {}'.format(next(my_generator)))
i = i+1
```
%% Cell type:markdown id: tags:
You should obtain the output:
```
Next Fibonacci number 0
Next Fibonacci number 1
Next Fibonacci number 1
Next Fibonacci number 2
Next Fibonacci number 3
Next Fibonacci number 5
Next Fibonacci number 8
Next Fibonacci number 13
Next Fibonacci number 21
Next Fibonacci number 34
```
While we have not avoided writing more code using the generator for the Fibonacci series, we have now separated the computation from the loop.
This allows us to further modularise the code and make it more flexible - and easier to debug. Also, conceptionally we have separted the actual computation from the way we use it.
%% Cell type:markdown id: tags:
# Decorators
Decorators are "higher-level functions" - they are functions that operate on functions. \
This allows them to change the behaviour of the function without modifying the function itself.
There are many situations where this might be useful. For example, you might want to add additional print statements for debugging purpuses without clogging up your code, check the type of variables you pass to the function without changing the function itself, you might want to time how long the execution of a function takes, how much memory it consumes, etc.
Besides, there are software packages that, for example, speed up your code without you having to modify it.
The general syntax is:
```
def my_decorator(func):
def wrapper(*args, **kwargs):
# do something before we call the function
result = func(*args, **kwargs)
# do something after the function returns
return result
return wrapper
```
We note:
* We define the decorator as an outer function that takes the function we want to manipulate as an argument.
* The constructs ```*args, **kwargs``` pass the arguments through the decorator to the function. Remeber that we do not know how many and which arguments we might have, hence we have to use the construct with the single and double asterix.
%% Cell type:code id: tags:
``` python
# define the decorator
def my_decorator(func):
def wrapper(*args, **kwargs):
print('Before the function starts')
result = func(*args, **kwargs)
print('After the function ends')
return result
return wrapper
```
%% Cell type:code id: tags:
``` python
# define the function
def sum(a,b):
return a + b
# execute the function normally:
c = sum(1,2)
print('The sum is {}'.format(c))
```
%% Output
The sum is 3
%% Cell type:markdown id: tags:
Now we use the decorator. There are two ways of calling the decorator:
* We can "wrap" the function by "replacing" it with the call to the decorator with the function as an argument
* We use the ```@``` symbol with the name of the decorator when we define the function.
The first approach is a little "clunky" as it adds a bit of code...
%% Cell type:code id: tags:
``` python
# Method 1
# define the function
def sum(a,b):
return a + b
sum = my_decorator(sum)
c = sum(1,2)
print('The sum is {}'.format(c))
```
%% Output
Before the function starts
After the function ends
The sum is 3
%% Cell type:code id: tags:
``` python
# Method 2
@my_decorator
def sum(a,b):
return a + b
c = sum(1,2)
print('The sum is {}'.format(c))
```
%% Output
Before the function starts
After the function ends
The sum is 3
%% Cell type:markdown id: tags:
Here you can see that we modify the behaviour of the function by "just" adding the decorator which has been defined elsewhere.
We do not modify the function itself, the only difference is that we use the ```@my_decorator```.
This can lead to interesting behaviour...
***Exercise***\
Write a decorator for the sum that intercepts the result and adds 10.
%% Cell type:code id: tags:
``` python
# ... your code here ...
```
......
pythonintro/solutions/Solutions_Python.ipynb
+ 149
− 0
View file @ 192971df
%% Cell type:code id: tags:
``` python
##
## print the word 'run' from the string 'nurse'
##
my_word = 'nurse'
print(my_word[2::-1])
```
%% Output
run
%% Cell type:markdown id: tags:
---
## Fibonacci numbers
%% Cell type:code id: tags:
``` python
##
## Fibonacci Numbers --- For Loop
##
Fibonacci = [0, 1]
for i in range(2,10,1):
Fib = Fibonacci[i-1] + Fibonacci[i-2]
Fibonacci.append(Fib)
print('The Fibonacci numbers are: {}'.format(Fibonacci))
```
%% Output
The Fibonacci numbers are: [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
%% Cell type:code id: tags:
``` python
##
## Fibonacci Numbers --- While Loop
##
Fibonacci = [0, 1]
while len(Fibonacci) < 10:
Fib = Fibonacci[-1] + Fibonacci[-2]
Fibonacci.append(Fib)
print('The Fibonacci numbers are: {}'.format(Fibonacci))
```
%% Cell type:code id: tags:
``` python
##
## Fibonacci Numbers --- Function
##
def compute_Fibonacci(n_numbers):
return_list = [0, 1]
for i in range(2,n_numbers,1):
Fib = return_list[i-1] + return_list[i-2]
return_list.append(Fib)
return return_list
Fib = compute_Fibonacci(n_numbers=10)
print('The Fibonacci numbers are: {}'.format(Fib))
```
%% Output
The Fibonacci numbers are: [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
%% Cell type:code id: tags:
``` python
##
## Fibonacci Numbers --- Generator
##
def fibonacci():
x = 0
y = 1
fib = 0
while True:
yield fib
fib = y + x
x = y
y = fib
my_generator = fibonacci()
i = 0
while i < 10:
print('Next Fibonacci number {}'.format(next(my_generator)))
i = i+1
```
%% Output
Next Fibonacci number 0
Next Fibonacci number 1
Next Fibonacci number 2
Next Fibonacci number 3
Next Fibonacci number 5
Next Fibonacci number 8
Next Fibonacci number 13
Next Fibonacci number 21
Next Fibonacci number 34
Next Fibonacci number 55
%% Cell type:markdown id: tags:
Python also allows to write multiple assignments on the same line, while during this assingment the value of the variables remains the same.
This allows for a slightly more efficient code:
%% Cell type:code id: tags:
``` python
##
## Fibonacci Numbers --- Generator (alternative)
##
def fibonacci():
# same as above - first assignment is x = 0, second assignment is y = 1
x, y = 0, 1
while True:
yield x
# note here we need one variable less. We are assigning x = y and y = x + y at the same time, so their values do not change
# Therefore, we do not need an intermediate variable in between.
x, y = y, x + y
my_generator = fibonacci()
i = 0
while i < 10:
print('Next Fibonacci number {}'.format(next(my_generator)))
i = i+1
```
%% Output
Next Fibonacci number 0
Next Fibonacci number 1
Next Fibonacci number 1
Next Fibonacci number 2
Next Fibonacci number 3
Next Fibonacci number 5
Next Fibonacci number 8
Next Fibonacci number 13
Next Fibonacci number 21
Next Fibonacci number 34
%% Cell type:markdown id: tags:
---
## Decorator
%% Cell type:code id: tags:
``` python
# define the decorator
def my_decorator(func):
def wrapper(*args, **kwargs):
result = func(*args, **kwargs)
result = result + 10
return result
return wrapper
@my_decorator
def sum(a,b):
return a + b
c = sum(1,2)
print('The sum is {}'.format(c))
```
%% Output
The sum is 13
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment