Skip to content

Conversation

@Reskov
Copy link

@Reskov Reskov commented Jun 21, 2021

  • removed double iteration over self.columns at the _do_load method
  • prepare the property to column mapping only when the column value was changed
  • removed extra loop over row values to check if all values are none
@Reskov Reskov force-pushed the optimize-do-load branch from 501f344 to e766dd3 Compare June 22, 2021 02:20
@Reskov
Copy link
Author

Reskov commented Jun 22, 2021

It is still pretty slow, but results are better on my laptop

@contextmanager def timer(): real_time = time.time() cpu_time = time.process_time() yield real_time = time.time() - real_time cpu_time = time.process_time() - cpu_time print(f"CPU time: {cpu_time:.2f} s, Real time: {real_time:.2f} s") async def test_model_alternative(user, event_loop): row = await User.query.gino.load(lambda r, c: r).first() # async def new_load(): loader = ModelLoader(User) ctx = {} with timer(): print("Full model") timeit.timeit(lambda: loader.do_load(row, ctx), number=20000) loader = ModelLoader(User, User.nickname) with timer(): print("Single column loader") timeit.timeit(lambda: loader.do_load(row, ctx), number=20000) row = await db.select([User.nickname]).gino.load(lambda r, c: r).first() loader = ModelLoader(User) with timer(): print("Partial loader") timeit.timeit(lambda: loader.do_load(row, ctx), number=20000)

Before:

Full model CPU time: 1.01 s, Real time: 1.05 s Single column loader CPU time: 0.47 s, Real time: 0.49 s Partial loader CPU time: 0.74 s, Real time: 0.75 s 

After:

Full model CPU time: 0.60 s, Real time: 0.60 s Single column loader CPU time: 0.37 s, Real time: 0.37 s Partial loader CPU time: 0.49 s, Real time: 0.49 s 
@wwwjfy
Copy link
Member

wwwjfy commented Jul 4, 2021

Looks good, thanks!

@wwwjfy wwwjfy merged commit 75fab19 into python-gino:master Jul 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants